popEye Documentation¶
About popEye¶
popEye is an IT integration tool. It can mapp diverse SQL DBs and No SQL DB, extract data from one source to another and maintain business logic in and easy SQL/python format.
popEye’s main goals are creating fast integration cycles and enabling to implement new requirement in a fast manner. It also provide a platform that enable to test and update business logic using the best of SQL in one hand and python on the other.
We believe that integration can be fast and much more simple buy using simple scripting language buy implementing 2 major concept :
- Fast design - the mapping module enable to create / maintance data model from scratch or by using existing data model. For example : to create full DWH implementation in Vertica based on mongo - all we have to do is create DWH model (entities, fields) using simple JSON format. popEye will manage all data type strucure and internal Vertica objects based on Mongo structure
- Fast Extract and loading - Loader module enable to full load / merge / increment methods for loading data on a scheduled process based on JSN defined mapping structure
This documentation is the first version of popEye, we do look for your help and we will provide our “wish-list” .
PopEye hope to extend and be one of the major open-source integration platform. come and join us
configuration¶
config instructions …..
Mapping module¶
Mapping module is use to create target structure based on source systems strucute or by define strucure in a Json format.
- Example of usage
assuming we are using oracle as source and Sql-server as target :
- create target strucure based on source query
- query is : “Select field1 As Yoyo, field2 as bobo … From oracle.table1” result
Jason mapping params¶
sample json mapping
Mapping samples¶
sample mapping