《自在城》房地产经典广告提案 平面设计.ppt

Introduction of ETL AutomationVersion
By Jet Wu, Consultant
NCR Copyright© 2001-2002
What is ETL Automation?
In Data Warehouse, ETL means Extraction, Transformation, Loading
Extraction
Extract data from the source database into a flat data file.
It occurs in external source environment.
Transformation
Transform data from flat data file to different format or from one table to another table in different data type.
It may occur in external source or in DW, usually in DW environment.
Loading
Load the data from flat data file into Data Warehouse.
It occurs in DW environment.
Sometimes, it is called ELT.
Automation Scheme for ETL
How do we make it automatically?
2018/5/2
2
NCR Confidential
ETL Automation Structure
2018/5/2
3
NCR Confidential
ETL Automation – Multiple Servers Support
Jobs at one server can trigger jobs at another server.
For example, once a group of data loading jobs were done at loading server, the cube job will be triggered at another OLAP server by ETL Automation automatically.
2018/5/2
4
NCR Confidential
ETL Automation – Multiple Servers Support
2018/5/2
5
NCR Confidential
ETL Automation Process
2018/5/2
6
NCR Confidential
Directory Tree in ETL Automation Server
/ETL
|--/APP
|-----/$SYS
|---------/$JOB
|-------------/bin
|-------------/ddl
|--/DATA
|-----/complete
|-----/fail
|--------/bypass
|--------/corrupt
|--------/duplicate
|--------/error
|--------/unknown
|-----/process
|-----/queue
|-----/receive
|--/LOG
|-----/$SYS
|--------/$DATE
|--/bin
|--/etc
|--/lock
|--/tmp
2018/5/2
7
NCR Confidential
ETL Automation Server Program
ETL Automation Server Programs are written in Perl







2018/5/2
8
NCR Confidential
Daemon Program in ETL Automation





2018/5/2
9
NCR Confidential
Daemon Program –
This daemon program will listen to a socket por