Introducing leveldb-server
We just released leveldb-server. Start forking: Github leveldb-server
leveldb-server
- Async leveldb server and client based on zeromq
- Storage engine leveldb
- Networking library zeromq
- We use leveldb-server at Safebox
License
New BSD license. Please see license.txt for more details.
Features
- Very simple key-value storage
- Data is sorted by key – allows
rangequeries - Data is automatically compressed
- Can act as persistent cache
- For our use at Safebox it replaced memcached+mysql
- Simple backups
cp -rf level.db backup.db - Networking/wiring from
zeromqmessaging library – allows many topologies - Async server for scalability and capacity
- Sync client for easy coding
- Easy polyglot client bindings. See zmq bindings
>>> db.put("k3", "v3")
'True'
>>> db.get("k3")
'v3'
>>> db.range()
'[{"k1": "v1"}, {"k2": "v2"}, {"k3": "v3"}]'
>>> db.range("k1", "k2")
'[{"k1": "v1"}, {"k2": "v2"}]'
>>> db.delete('k1')
>>>
Will be adding high availability, replication and autosharding using the same zeromq framework.
Dependencies
python 2.6+ (older versions with simplejson) zmq pyzmq leveldb pyleveldb
Getting Started
Instructions for an EC2 Ubuntu box.
Installing zeromq
wget http://download.zeromq.org/zeromq-2.1.10.tar.gz tar xvfz zeromq-2.1.10.tar.gz cd zeromq-2.1.10 sudo ./configure sudo make sudo make install
Installing pyzmq
wget https://github.com/zeromq/pyzmq/downloads/pyzmq-2.1.10.tar.gz tar xvfz pyzmq-2.1.10.tar.gz cd pyzmq-2.1.10/ sudo python setup.py configure --zmq=/usr/local/lib/ sudo python setup.py install
Installing leveldb and pyleveldb
svn checkout http://py-leveldb.googlecode.com/svn/trunk/ py-leveldb-read-only cd py-leveldb-read-only sudo compile_leveldb.sh sudo python setup.py install
Starting the leveldb-server
> python leveldb-server.py -h Usage: leveldb-server.py -p [port and host settings] Default: tcp://127.0.0.1:5147 -f [database file name] Default: level.db leveldb-server Options: --version show program's version number and exit -h, --help show this help message and exit -p HOST, --host=HOST -d DBFILE, --dbfile=DBFILE > python leveldb-server.py
Using the leveldb-client-py
> cd clients/py/
> sudo python setup.py install
> python
>>> from leveldbClient import database
>>> db = database.leveldb()
>>> db.get("Key")
>>> db.put("K", "V")
>>> db.range()
>>> db.range(start, end)
>>> db.delete("K")
Backups
cp -rpf level.db backup.db
Known issues and work in progress
Would love your pull requests on
- Benchmarking and performance analysis
- client libraries for other languages
- [issue] zeromq performance issues with 1M+ inserts at a time
- [feature] timeouts in client library
- [feature] support for counters
- [feature] limit support in range queries
- Serializing and seperate threads for get/put/range in leveldb-server
- HA/replication/autosharding and possibly pub-sub for replication
Thanks
Thanks to all the folks who have contributed to all the dependencies. Special thanks to pyzmq/examples/mongo* author for inspiration.
