Skip to content

Milestones

List view

  • Basic structure

    Establishing connection between the server node and the working node via SOCKET. - Python object serialization - Load test - logging - Server and client side communication

    Overdue by 12 year(s)
    •
    Due by September 22, 2013
    •4/5 issues closed
    80% complete1 open4 closed

  • Documentation

    Final milestone. Should take place through out the whole project's duration.

    Overdue by 12 year(s)
    •
    Due by November 30, 2013
    •7/7 issues closed
    100% complete0 open7 closed

  • Data storage

    Data storage (MongoDB) -More to come-

    Overdue by 12 year(s)
    •
    Due by November 11, 2013
    •1/2 issues closed
    50% complete1 open1 closed

  • Python data capture rules deployment

    Deploy the data capture rule file from the server node to the working nodes -More to come-

    Overdue by 12 year(s)
    •
    Due by October 31, 2013
    •2/2 issues closed
    100% complete0 open2 closed

  • Scrapping module

    Scraping module for working nodes -More to come-

    Overdue by 12 year(s)
    •
    Due by October 21, 2013
    •3/3 issues closed
    100% complete0 open3 closed

  • Crawling module

    Crawling module for working nodes -More to come-

    Overdue by 12 year(s)
    •
    Due by October 14, 2013
    •4/4 issues closed
    100% complete0 open4 closed

  • URL dispatching / URL aggregation

    Server side : URL dispatching to the working nodes Working node side: URL aggregation and transmission to the server node

    Overdue by 12 year(s)
    •
    Due by October 7, 2013
    •5/5 issues closed
    100% complete0 open5 closed

  • Communication protocol (node-to-node)

    Communication protocol between the master node and the working nodes. Information that needs to be transported : - General status updates - Dispatching URLs - Return scrapped data - Return scrapped URLs

    Overdue by 12 year(s)
    •
    Due by September 30, 2013
    •5/5 issues closed
    100% complete0 open5 closed