/cloud-crowd

Parallel Processing for the Rest of Us
  1. Ruby 86.7%
  2. JavaScript 6.1%
  3. CSS 4.5%
  4. HTML 2.7%
Ruby JavaScript CSS HTML
Switch branches/tags
Nothing to show
Find file
Clone or download

Clone with HTTPS

Use Git or checkout with SVN using the web URL.

Download ZIP

Launching GitHub Desktop...

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop...

If nothing happens, download GitHub Desktop and try again.

Launching Xcode...

If nothing happens, download Xcode and try again.

Launching Visual Studio...

If nothing happens, download the GitHub extension for Visual Studio and try again.

Image Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
actions adding an artificial slowdown to the word count example, for demonstr… Dec 5, 2009
bin moving things around in an attempt to get the raw app started -- rake… Aug 23, 2009
config Black list table being added Aug 19, 2015
examples ready to merge back to master... Sep 16, 2009
lib Extract configuration logic into methods that take hashes not just pa… Mar 23, 2018
public Cleaning up css for blacklisted actions Aug 26, 2015
test Replace FactoryGirl with FactoryBot Mar 23, 2018
views Admin console is now bringing in real blacklist data Aug 26, 2015
wiki CloudCrowd 0.6.2 Apr 14, 2011
.gitignore update gitignore. May 6, 2014
.yardopts keeping yardopts for yard generation Sep 15, 2009
EPIGRAPHS removing some old examples Aug 27, 2009
Gemfile Replace FactoryGirl with FactoryBot Mar 23, 2018
Gemfile-ar32 A testing Gemfile with ActiveRecord locked to 3.2 Jun 5, 2014
Gemfile-ar32.lock A testing Gemfile with ActiveRecord locked to 3.2 Jun 5, 2014
Gemfile.lock Replace FactoryGirl with FactoryBot Mar 23, 2018
LICENSE back to the MIT license, because that's what I promised rubyforge Aug 27, 2009
README Update README links to wiki/rdoc Dec 21, 2016
Rakefile Bumping things. May 20, 2017
TODO clearing todo list Oct 19, 2009
cloud-crowd.gemspec Consolidate VERSION in one location Jun 26, 2014

README 

=                                                                               
           _  _                                                                
          ( `   )_                                                             
         (    )    `)                                                          
       (_   (_ .  _) _)                                                        
                                      _                                        
                                     (  )                                      
      _ .                         ( `  ) . )                                   
    (  _ )_                      (_, _(  ,_)_)                                 
  (_  _(_ ,)                                                                   
                                                                               
           _  _               ___ _             _  ___                   _     
          ( `   )_           / __| |___ _  _ __| |/ __|_ _ _____ __ ____| |    
         (    )    `)       | (__| / _ \ || / _` | (__| '_/ _ \ V  V / _` |    
       (_   (_ .  _) _)      \___|_\___/\_,_\__,_|\___|_| \___/\_/\_/\__,_|    
                                                                               
                                                     _                         
                                                    (  )                       
                  _, _ .                         ( `  ) . )                    
                 ( (  _ )_                      (_, _(  ,_)_)                  
               (_(_  _(_ ,)                                                    
                                                                               
                                                                               
                                                                               
  ~ CloudCrowd ~

    * Parallel processing for the rest of us
    * Write your scripts in Ruby
    * Works with Amazon EC2 and S3
    * split -> process -> merge
    * As easy as `gem install cloud-crowd`

    Well-suited for:
    
    * Generating or resizing images.
    * Encoding video.
    * Running text extraction or OCR on PDFs.
    * Migrating a large file set or database.
    * Web scraping.
    
    
  ~ Documentation ~
  
    Wiki: https://github.com/documentcloud/cloud-crowd/wiki
    Rdoc: http://www.rubydoc.info/github/documentcloud/cloud-crowd
  
  
  ~ Getting started ~
  
    # Install the gem.
    
      >> sudo gem install cloud-crowd
    
    # Install the CloudCrowd configuration files to a location of your choosing.
    
      >> crowd install ~/config/cloud-crowd
    
    # Now, you can use the full complement of `crowd` commands from inside of
    # this configuration directory. To see the available commands:
    
      >> crowd --help
    
    # Edit the configuration files to your satisfaction, add AWS credentials, 
    # and then load the CloudCrowd schema into your configured database.
    
      >> cd ~/config/cloud-crowd
      >> mate config.yml
      >> mate database.yml
      >> [create the database you just configured...]
      >> crowd load_schema
    
    # Write your actions, and install them into the 'actions' subdirectory.
    # CloudCrowd comes with a few default actions as an example.
    
    # To launch the central server (make sure that you include its location
    # in config.yml):
    
      >> crowd server
    
    # The configuration folder also includes 'config.ru', which can be used by
     # any Rack-compliant webserver to run your central server.
    
    # Then, to launch a node of workers:
    
      >> crowd node
    
    # To spin up remote nodes, install the 'cloud-crowd' gem and copy over
    # your configuration directory. Run `crowd node`, and the remote machines
    # will register with the central server, becoming available for processing.
    
    # At this point you can visit your Operations Center at localhost:9173 to 
    # view all of your nodes, ready for action.