MySQL to PostgreSQL Replica

Pg_chameleon is a replication tool from MySQL to PostgreSQL developed in Python 2.7 and Python 3.3+





The system uses the mysql-replication library to pull the changes from MySQL and covert them into a jsonb object.
A plpgsql function decodes the jsonb and replays the changes into the PostgreSQL database.

The github repository is available here https://github.com/the4thdoctor/pg_chameleon/ 

The replica system is now stable and is available on Pypi.

The documentation is available here 

The replica initialisation  pulls the data from MySQL locking the database in read only mode using FLUSH TABLE WITH READ LOCK; .

It's possible to pull the data from a MySQL  cascading replica if the MySQL slave is configured with log-slave-updates.

Installation in virtualenv

The system is designed to work within a virtualenv. Is still possible to install the library and script system wide though.

No daemon

The replica should be started using a cron job. As the replica detects if another process is running is safe to have a periodic start (e.g. every 30 minutes) to keep the script running in the case of crash.


DDL replica limitations

DDL and DML mixed in the same transaction are not decoded in the right order. This can result in a replica issues caused by a wrong jsonb structure, should the DML change the data on the same table modified by the DDL.

Full changelog available here

No comments:

Post a Comment