Blog

MongoDB – How To Benchmark and Test Performance Using mongo-perf

January 16, 2012 9:36 pm / by / 3 comments

Benchmarking MongoDB

Here at BeyondFog, we are very interested in how MongoDB performs – and how to make it faster! While performance testing in general is a tricky thing, depending heavily upon the details of your application, some scenarios are easy enough to benchmark. Imagine you wished to compare MongoDB performance on the EXT4FS filesystem with MongoDB performance on the XFS filesystem. A tool which can generate reproducible load on the database and record response time is perfect for this situation. This kind of tool is called a synthetic benchmark.

Mongo-Perf

Mongo Perf Graph

Fortunately, such a synthetic benchmarking tool already exists for MongoDB in the form of 10gen’s mongo-perf utility. Less fortunately, this tool is not very well documented and can be a little tricky to get running. This article hopes to help you set it up.

mongo-perf comprises a C++ program and a couple of Python harnesses. The C++ program needs to be linked against the mongoclient library, which is not included in standard MongoDB binary distributions. For this reason, you’ll need to build MongoDB from source, enabling the mongoclient headers and library binaries. In this blog post, we will provide instructions for doing this under Mac OS X 10.7 (Lion) and Ubuntu 11.10 (Oneric).

Getting Mongo-Perf Dependencies for Mac OS X 10.7

There are three major dependencies required to build MongoDB on OS X: XCode, the Boost family of C++ libraries and the SCons build tool. XCode is available from Apple. Both Boost and SCons are installable via MacPorts and Homebrew. It is your choice which package manager to use.

Option A – Mac Ports

To install MacPorts simply download the Lion installer and run it. For more detailed instructions see the MacPorts install page. Once you have MacPorts installed, you can simply run:

port install boost scons

Option B – Homebrew

For complete Homebrew install instructions, see this Wiki page. The short version is to run the command:

/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"

Once you have Homebrew, you can install Boost and SCons with the commands:

brew install boost; brew install scons

Getting Mongo-Perf Dependencies for Ubuntu 11.10

On Ubuntu, you can simply use apt-get to install the requisite packages. From 10gen’s Linux build page:

apt-get -y install tcsh git-core scons g++
apt-get -y install libpcre++-dev libboost-dev libreadline-dev
apt-get -y install libboost-program-options-dev libboost-thread-dev libboost-filesystem-dev libboost-date-time-dev

Building MongoDB

Now that you have the various MongoDB dependencies on your system, you can build MongoDB itself. The first step is checking out the source code from Github:

git clone https://github.com/mongodb/mongo.git

Note: You could just as easily use a source tarball from http://www.mongodb.org/downloads.

Now you should be able to cd into the root of the source directory and build it. Before building it, you should choose a prefix. We recommend /opt/local as it simplifies building mongo-perf itself on OS X later. We have some shell magic in the scons snippets below to automatically add the –prefix=/opt/local on OS X as needed.

cd mongo; scons $(test $(uname) = "Darwin" && echo -n --prefix=/opt/local) --full all

Depending on how fast your machine is, this could take some time. We suggest using this as an opportunity to prepare a beverage. Once this has completed successfully, install the binaries, headers, libraries and so on using the command:

sudo scons $(test $(uname) = "Darwin" && echo -n --prefix=/opt/local) --full install

Building Mongo-Perf

Now you are ready to build Mongo-Perf. To fetch the mongo-perf source from GitHub, run:

git clone https://github.com/mongodb/mongo-perf.git

If all the dependencies were installed in the correct places, you should be able to build immediately:

cd mongo-perf; scons

To run Mongo-Perf, you will need a Python environment with the pymongo driver available. The easiest way to set this up is to create a “virtualenv” and install pymongo into it. To get the virtualenv tool, simply download it from https://raw.github.com/pypa/virtualenv/master/virtualenv.py.

wget --no-check-certificate https://raw.github.com/pypa/virtualenv/master/virtualenv.py

Now to create a virtualenv and install pymongo, run:

python virtualenv.py env
source env/bin/activate
pip install pymongo

Using Mongo-Perf

Mongo-Perf benchmarks are kicked off by the Python program “runner.py”. You can either launch this against an already-running MongoDB server on your local system, or you can have runner.py fetch and build a version of MongoDB from source (at any revision you want) and use that. There is also an option to run it against a mongos router to benchmark a sharded configuration.

For this article, we’ll assume you already have a MongoDB server running on localhost port 27017. If you don’t have MongoDB running locally just yet, see 10gen’s MongoDB Quickstart OS X for OS X install instructions or their Ubuntu and Debian Packages page for Ubuntu install instructions.

To start a Mongo-Perf benchmark against this, run:

python runner.py --nolaunch --port 27017

You should immediately see output from the benchmark tool about which load test is being run. Upon completion, the program will emit a JSON dump of the test results to the console, and also write it to the local MongoDB in a database called “bench_results”.

The easiest way to view the benchmark results is through the included web app, server.py. To start the web app, run:

python server.py

Once that is running, point your browser to http://localhost:8080 to view the results. As you will see, there are a number of different kinds of tests – inserts, reads and updates. The tests are run with varying levels of concurrency which helps evaluate performance with multiple clients.

That should be enough to get you started with your own synthetic MongoDB benchmarks. In future posts we will be talking about some of our results and how to tune your database for better performance. If you are interested in more cloud and database tips & tricks, sign up for our newsletter! Also, if you are using MongoDB in your business we’d love to hear from you.

Tags

 
  • Hans Cris

    Can you explain how I can run this on Windows??

  • http://twitter.com/pburkholder Peter Burkholder

    Thanks for helping me sidestep some frustration with this write up. For me on OSX, it seems the homebrew mongodb install includes the development files needed for the benchmark to build. The steps came to:

    brew install mongodb
    easy_install pymongo
    brew install scons
    brew install boost 
    git clone https://github.com/mongodb/mongo-perf.git
    cd mongo-perf/
    (cd mongo-cxx-driver/; scons)
    scons

    The last build issued an error related to /opt/local/lib, but seems to run.

  • http://davidvg.com/ David van Geest

    Thanks for the instructions! They worked well for me on Linux Mint, but I’m having trouble making them work on CentOS.

    I’ve built MongoDB, but mongo-perf is giving me trouble, specifically:

    /root/mongodb-src-r2.4.3/src/third_party/boost/boost/system/error_code.hpp:214: undefined reference to `boost::system::generic_category()’

    I do have the boost libraries installed, and they are referenced in the g++ command.

    More details here: http://stackoverflow.com/questions/16444366/undefined-boost-reference-when-compiling-mongo-perf-on-centos-6-3

    Any ideas would be appreciated!