OSDN -- Desenvolver e Download de Software Open Source
Estado tradução de Português translation status for pt

RSS
Download List

Projeto Descrição

Ebot is a scalable and distribuited Web crawler. The URLs are saved to a NOSQL database (which supports map/reduce queries) that you can query via RESTful HTTP requests or using your preferred programming languages. The URLs that need to be analyzed are sent to AMQP queues. In this way, it is possible to run several crawlers in parallel and stop and start them without losing URLs.

System Requirements

System requirement is not defined
Information regarding Project Releases and Project Resources. Note that the information here is a quote from Freecode.com page, and the downloads themselves may not be hosted on OSDN.

2010-12-30 13:03
0.9

A melhor arquitetura de plugin. Novos plugins para salvar a imagem URLs e títulos de páginas HTML.
A better plugin architecture. New plugins for saving image URLs and titles of HTML pages.

2010-12-18 18:59
0.8

Esta versão foi atualizada para os últimos lançamentos do RabbitMQ (2.2.0) e couchbeam. Alguns bugs foram corrigidos.
This release was updated to the latest releases of rabbitmq (2.2.0) and couchbeam. Some bugs were fixed.

2010-09-18 07:05
0.7

Compatibilidade com Erlang R14A. Testados com Debian Testing.
Compatibility with Erlang R14A. Tested with Debian Testing.

2010-08-27 07:24
0.6

Compatibilidade com os mais recentes (de desenvolvimento) a libertação de bibliotecas e software necessários (RabbitMQ, CouchDB, webmachine, riak, etc)
Compatibility with the latest (development) releases of required libraries and software (rabbitmq, couchdb, webmachine, riak, etc.)

2010-06-20 23:18
0.5

Para uma melhor escalabilidade, filas AMQP novas são usadas (ebot.new ebot.fetched .* .* .* ebot.completed ebot.refused .*) e do núcleo antigo do rastreador (ebot_web) está dividido em dois módulos distintos e paralelos / processos (ebot_html e ebot_web).
For better scalability, new amqp queues are used (ebot.new.* ebot.fetched.* ebot.completed.* ebot.refused.*) and the old core of the crawler (ebot_web) is now split in two different and parallel modules/processes (ebot_html and ebot_web).

Project Resources

Home Page Project Page