大数据方案与案例46.ppt

大数据方案与案例46
每天几百 GB、 几 TB 的资料，且持续成长中
储存
Storing
在收数据的同时做必要的前置处理 (pre-processing)
，并区分数据处理的优先等级 (/?m=index&f=view&p=24hour&s=nlife
/?mod=store&func=style_show&SR_NO=DEAO45

/



/?m

/?mod

…………………………
- - [15/Nov/2011:00:07:45] "GET /flower_store/?product_id=FL- 10"?category_id=PLANTS&JSESSIONID=SD7SL1FF9ADFF2" "Mozilla/ (X11; U; Linux i686; en-US; rv:) Gecko/20070223 CentOS/- Firefox/" 1604 - - [15/Nov/2011:00:07:58] "GET /flower_store/?product_id=K9-BD-01 HTTP/" "?category_id=FLOWERS&JSESSIONID=SD7SL1FF9ADFF2" "Mozilla/ (X11; U; Linux i686; en-US; rv:) Gecko/20070223 CentOS/- Firefox/" 1803 - - [15/Nov/2011:00:08:26] "GET /flower_store/?product_id=FL-DSH-01 HTTP/1 "?category_id=FLOWERS&JSESSIONID=SD7SL1FF9ADFF2" "Mozilla/ (X11; U; Linux i686; en-US; rv:) Gecko/20070223 CentOS/- Firefox/" 1025 -
IP
Action
Time
Server
URL
###
……….
………..
………
……….
……..
…….
……….
………..
………
……….
……..
…….
3rd party
pre-defined
Reporting
Filter
String Truncation
Log file : / day
什么是大数据处理 (大数据 Processing)
RDR_SEND_TIME
RDR_TAG
SUBSCRIBER_ID
PACKAGE_ID
SERVICE_ID
PROTOCOL_ID
SKIPPED_SESSIONS
SERVER_IP
SERVER_PORT
ACCESS_STRING
INFO_STRING
CLIENT_IP
CLIENT_PORT
INITIATING_SIDE
REPORT_TIME
MILLISEC_DURATION
P