Peter M. Groen / redox

Commit 9e898af2395000891678e7c3b550b8e54682d2ca

Authored by Hayk Martirosyan
1 parent 1a47846b

Add vector<string> reply type, benchmark info in README 

Also plan to add in set<string> and unordered_set<string>. Maybe
queue<string>. Looking at likely removing char* as an option, if it
shows to be about the same speed as string.
Inline Side-by-side
Showing 10 changed files with 236 additions and 31 deletions
CMakeLists.txt
  View file @9e898af
... ... @@ -35,8 +35,8 @@ target_link_libraries(basic ${LIB_REDIS})
35 35 add_executable(basic_threaded examples/basic_threaded.cpp ${SRC_ALL})
36 36 target_link_libraries(basic_threaded ${LIB_REDIS})
37 37  
38   -#add_executable(lpush_benchmark examples/lpush_benchmark.cpp ${SRC_ALL})
39   -#target_link_libraries(lpush_benchmark ${LIB_REDIS})
  38 +add_executable(lpush_benchmark examples/lpush_benchmark.cpp ${SRC_ALL})
  39 +target_link_libraries(lpush_benchmark ${LIB_REDIS})
40 40  
41 41 add_executable(speed_test_async examples/speed_test_async.cpp ${SRC_ALL})
42 42 target_link_libraries(speed_test_async ${LIB_REDIS})
... ... @@ -46,3 +46,9 @@ target_link_libraries(speed_test_sync ${LIB_REDIS})
46 46  
47 47 add_executable(speed_test_async_multi examples/speed_test_async_multi.cpp ${SRC_ALL})
48 48 target_link_libraries(speed_test_async_multi ${LIB_REDIS})
  49 +
  50 +add_executable(data_types examples/data_types.cpp ${SRC_ALL})
  51 +target_link_libraries(data_types ${LIB_REDIS})
  52 +
  53 +add_executable(string_v_char examples/string_vs_charp.cpp ${SRC_ALL})
  54 +target_link_libraries(string_v_char ${LIB_REDIS})
... ...
README.md
  View file @9e898af
... ... @@ -3,4 +3,92 @@ redox
3 3  
4 4 Modern, asynchronous, and wicked fast C++11 bindings for Redis
5 5  
6   -Work in progress, details coming soon.
  6 +## Overview
  7 +
  8 +Redox provides an elegant high-level interface to Redis that makes it easy to write
  9 +high-performance applications. It is built on top of [hiredis](https://github.com/redis/hiredis/)
  10 +(but uses only its asynchronous API, even for synchronous calls) and
  11 +[libev](http://manpages.ubuntu.com/manpages/raring/man3/ev.3.html).
  12 +
  13 + * A single `command()` method, templated by the user based on the expected return type
  14 + * Callbacks are [std::functions](http://en.cppreference.com/w/cpp/utility/functional/function),
  15 + so they can capture state - lambdas, member functions, bind expressions
  16 + * Thread-safe - use one Redox object in multiple threads or multiple Redox objects in one thread
  17 + * Automatic pipelining, even for synchronous calls from separate threads
  18 + * Access to low-level reply objects when needed
  19 + * 100% clean Valgrind reports
  20 +
  21 +## Performance Benchmarks
  22 +Benchmarks are given by averaging the results of five trials of various programs
  23 +in `examples/`. The test machine is a Samsung Series 9 with Ubuntu 14.04 (64-bit),
  24 +Intel i5-3317U @ 1.70GHz x 4, 7.5 GiB RAM, 120 GB SSD. During these tests, Redox
  25 +communicated with a local Redis server over TCP.
  26 +
  27 + * Blocking command in a loop (`speed_test_sync`): **36,250 commands/s**
  28 + * One repeating asynchronous command (`speed_test_async`): **158,849 commands/s**
  29 + * 100 repeating asynchronous commands (`speed_test_async_multi`): **608,032 commands/s**
  30 +
  31 +## Tutorial
  32 +Coming soon. For now, look at the example programs located in `examples/`, and the snippets
  33 +posted below.
  34 +
  35 +Basic synchronous usage:
  36 +
  37 + redox::Redox rdx = {"localhost", 6379}; // Initialize Redox
  38 + rdx.start(); // Start the event loop
  39 +
  40 + rdx.del("occupation");
  41 +
  42 + if(!rdx.set("occupation", "carpenter")) // Set a key, check if succeeded
  43 + cerr << "Failed to set key!" << endl;
  44 +
  45 + cout << "key = occupation, value = \"" << rdx.get("occupation") << "\"" << endl;
  46 +
  47 + rdx.stop(); // Shut down the event loop
  48 +
  49 +Output: `key = occupation, value = "carpenter"`
  50 +
  51 +The `command` method launches a command asynchronously. This is the root
  52 +of every method in Redox that executes a command:
  53 +
  54 + /**
  55 + * Create an asynchronous Redis command to be executed. Return a pointer to a
  56 + * Command object that represents this command. If the command succeeded, the
  57 + * callback is invoked with a reference to the reply. If something went wrong,
  58 + * the error_callback is invoked with an error_code. One of the two is guaranteed
  59 + * to be invoked. The method is templated by the expected data type of the reply,
  60 + * and can be one of {redisReply*, string, char*, int, long long int, nullptr_t}.
  61 + *
  62 + * cmd: The command to be run.
  63 + * callback: A function invoked on a successful reply from the server.
  64 + * error_callback: A function invoked on some error state.
  65 + * repeat: If non-zero, executes the command continuously at the given rate
  66 + * in seconds, until cancel() is called on the Command object.
  67 + * after: If non-zero, executes the command after the given delay in seconds.
  68 + * free_memory: If true (default), Redox automatically frees the Command object and
  69 + * reply from the server after a callback is invoked. If false, the
  70 + * user is responsible for calling free() on the Command object.
  71 + */
  72 + template<class ReplyT>
  73 + Command<ReplyT>* command(
  74 + const std::string& cmd,
  75 + const std::function<void(const std::string&, const ReplyT&)>& callback = nullptr,
  76 + const std::function<void(const std::string&, int status)>& error_callback = nullptr,
  77 + double repeat = 0.0,
  78 + double after = 0.0,
  79 + bool free_memory = true
  80 + );
  81 +
  82 +The `command_blocking` method is the root of all synchronous calls. It calls `command` then
  83 +uses a condition variable to wait for a reply.
  84 +
  85 + /**
  86 + * A wrapper around command() for synchronous use. Waits for a reply, populates it
  87 + * into the Command object, and returns when complete. The user can retrieve the
  88 + * results from the Command object - ok() will tell you if the call succeeded,
  89 + * status() will give the error code, and reply() will return the reply data if
  90 + * the call succeeded.
  91 + */
  92 + template<class ReplyT>
  93 + Command<ReplyT>* command_blocking(const std::string& cmd);
  94 +
... ...
examples/basic.cpp
  View file @9e898af
... ... @@ -9,13 +9,15 @@ using namespace std;
9 9  
10 10 int main(int argc, char* argv[]) {
11 11  
12   - redox::Redox rdx; // Initialize Redox (default host/port)
  12 + redox::Redox rdx = {"localhost", 6379}; // Initialize Redox
13 13 rdx.start(); // Start the event loop
14 14  
15   - if(!rdx.set("alaska", "rules")) // Set a key, check if succeeded
  15 + rdx.del("occupation");
  16 +
  17 + if(!rdx.set("occupation", "carpenter")) // Set a key, check if succeeded
16 18 cerr << "Failed to set key!" << endl;
17 19  
18   - cout << "key = alaska, value = " << rdx.get("alaska") << endl;
  20 + cout << "key = occupation, value = \"" << rdx.get("occupation") << "\"" << endl;
19 21  
20 22 rdx.stop(); // Shut down the event loop
21 23 }
... ...
examples/data_types.cpp 0 → 100644
  View file @9e898af
  1 +/**
  2 +* Basic use of Redox to set and get a Redis key.
  3 +*/
  4 +
  5 +#include <iostream>
  6 +#include "../src/redox.hpp"
  7 +
  8 +using namespace std;
  9 +
  10 +int main(int argc, char* argv[]) {
  11 +
  12 + redox::Redox rdx; // Initialize Redox (default host/port)
  13 + rdx.start(); // Start the event loop
  14 +
  15 + rdx.del("mylist");
  16 +
  17 + rdx.command_blocking("LPUSH mylist 1 2 3 4 5 6 7 8 9 10");
  18 +
  19 + auto c = rdx.command_blocking<vector<string>>("LRANGE mylist -3 -1");
  20 + if(!c->ok()) cerr << "Error with LRANGE: " << c->status() << endl;
  21 + for(const string& s : c->reply()) cout << s << endl;
  22 + c->free();
  23 +
  24 +// if(!rdx.set("apples", "are great!")) // Set a key, check if succeeded
  25 +// cerr << "Failed to set key!" << endl;
  26 +//
  27 +// cout << "key = alaska, value = \"" << rdx.get("apples") << "\"" << endl;
  28 +
  29 + rdx.stop(); // Shut down the event loop
  30 +}
... ...
examples/lpush_benchmark.cpp
  View file @9e898af
... ... @@ -7,44 +7,45 @@
7 7  
8 8 using namespace std;
9 9  
10   -unsigned long time_ms() {
11   - return chrono::system_clock::now().time_since_epoch()
12   - /chrono::milliseconds(1);
  10 +double time_s() {
  11 + unsigned long ms = chrono::system_clock::now().time_since_epoch() / chrono::microseconds(1);
  12 + return (double)ms / 1e6;
13 13 }
14 14  
15 15 int main(int argc, char* argv[]) {
16 16  
17   - redox::Redox rdx = {"localhost", 6379};
  17 + redox::Redox rdx;
18 18 rdx.start();
19 19  
20   - // TODO wait for this somehow
21   - rdx.command("DEL test");
  20 + rdx.del("test");
22 21  
23   - unsigned long t0 = time_ms();
24   - unsigned long t1 = t0;
  22 + double t0 = time_s();
  23 + double t1 = t0;
25 24  
26 25 int len = 1000000;
27   - int count = 0;
  26 + atomic_int count = {0};
28 27  
29 28 for(int i = 1; i <= len; i++) {
30 29 rdx.command<int>("lpush test 1", [&t0, &t1, &count, len, &rdx](const string& cmd, int reply) {
31 30  
32   - count++;
  31 + count += 1;
  32 +
33 33 if(count == len) {
34 34 cout << cmd << ": " << reply << endl;
35 35  
36   - unsigned long t2 = time_ms();
37   - cout << "Time to queue async commands: " << t1 - t0 << "ms" << endl;
38   - cout << "Time to receive all: " << t2 - t1 << "ms" << endl;
39   - cout << "Total time: " << t2 - t0 << "ms" << endl;
  36 + double t2 = time_s();
  37 + cout << "Time to queue async commands: " << t1 - t0 << "s" << endl;
  38 + cout << "Time to receive all: " << t2 - t1 << "s" << endl;
  39 + cout << "Total time: " << t2 - t0 << "s" << endl;
  40 + cout << "Result: " << (double)len / (t2-t0) << " commands/s" << endl;
40 41  
41   - rdx.stop();
  42 + rdx.stop_signal();
42 43 }
43 44 });
44 45 }
45   - t1 = time_ms();
  46 + t1 = time_s();
46 47  
47   - rdx.block_until_stopped();
  48 + rdx.block();
48 49  
49 50 cout << "Commands processed: " << rdx.num_commands_processed() << endl;
50 51  
... ...
examples/speed_test_async_multi.cpp
  View file @9e898af
... ... @@ -21,7 +21,7 @@ int main(int argc, char* argv[]) {
21 21 Redox rdx = {"localhost", 6379};
22 22 rdx.start();
23 23  
24   - if(rdx.command_blocking("SET simple_loop:count 0")) {
  24 + if(rdx.set("simple_loop:count", "0")) {
25 25 cout << "Reset the counter to zero." << endl;
26 26 } else {
27 27 cerr << "Failed to reset counter." << endl;
... ... @@ -55,9 +55,7 @@ int main(int argc, char* argv[]) {
55 55 for(auto c : commands) c->cancel();
56 56  
57 57 // Get the final value of the counter
58   - auto get_cmd = rdx.command_blocking<string>("GET simple_loop:count");
59   - long final_count = stol(get_cmd->reply());
60   - get_cmd->free();
  58 + long final_count = stol(rdx.get("simple_loop:count"));
61 59  
62 60 rdx.stop();
63 61  
... ...
examples/string_vs_charp.cpp 0 → 100644
  View file @9e898af
  1 +/**
  2 +* Redox test
  3 +* ----------
  4 +* Increment a key on Redis using an asynchronous command on a timer.
  5 +*/
  6 +
  7 +#include <iostream>
  8 +#include "../src/redox.hpp"
  9 +
  10 +using namespace std;
  11 +using namespace redox;
  12 +
  13 +double time_s() {
  14 + unsigned long ms = chrono::system_clock::now().time_since_epoch() / chrono::microseconds(1);
  15 + return (double)ms / 1e6;
  16 +}
  17 +
  18 +int main(int argc, char* argv[]) {
  19 +
  20 + Redox rdx;
  21 + rdx.start();
  22 +
  23 + rdx.del("stringtest");
  24 + rdx.set("stringtest", "value");
  25 +
  26 + int count = 1000000;
  27 + double t0 = time_s();
  28 +
  29 + string cmd_str = "GET stringtest";
  30 + for(int i = 0; i < count; i++) {
  31 + rdx.command<string>(
  32 + cmd_str,
  33 + [](const string &cmd, string const& value) {
  34 + value;
  35 + },
  36 + [](const string &cmd, int status) {
  37 + cerr << "Bad reply: " << status << endl;
  38 + }
  39 + );
  40 + }
  41 +
  42 + rdx.stop();
  43 +
  44 + double t_elapsed = time_s() - t0;
  45 +
  46 + cout << "Sent " << count << " commands in " << t_elapsed << "s." << endl;
  47 +
  48 + return 0;
  49 +}
... ...
src/command.cpp
  View file @9e898af
... ... @@ -2,6 +2,9 @@
2 2 * Redis C++11 wrapper.
3 3 */
4 4  
  5 +#include <vector>
  6 +#include <assert.h>
  7 +
5 8 #include "command.hpp"
6 9  
7 10 namespace redox {
... ... @@ -106,4 +109,29 @@ void Command&lt;std::nullptr_t&gt;::invoke_callback() {
106 109 }
107 110 }
108 111  
  112 +
  113 +template<>
  114 +void Command<std::vector<std::string>>::invoke_callback() {
  115 +
  116 + if(is_error_reply()) invoke_error(REDOX_ERROR_REPLY);
  117 +
  118 + else if(reply_obj->type != REDIS_REPLY_ARRAY) {
  119 + std::cerr << "[ERROR] " << cmd << ": Received non-array reply." << std::endl;
  120 + invoke_error(REDOX_WRONG_TYPE);
  121 +
  122 + } else {
  123 + std::vector<std::string> v;
  124 + size_t count = reply_obj->elements;
  125 + for(size_t i = 0; i < count; i++) {
  126 + redisReply* r = *(reply_obj->element + i);
  127 + if(r->type != REDIS_REPLY_STRING) {
  128 + std::cerr << "[ERROR] " << cmd << ": Received non-array reply." << std::endl;
  129 + invoke_error(REDOX_WRONG_TYPE);
  130 + }
  131 + v.push_back(r->str);
  132 + }
  133 + invoke(v);
  134 + }
  135 +}
  136 +
109 137 } // End namespace redox
... ...
src/redox.cpp
  View file @9e898af
... ... @@ -112,7 +112,7 @@ void Redox::run_event_loop() {
112 112  
113 113 if(commands_created != commands_deleted) {
114 114 cerr << "[ERROR] All commands were not freed! "
115   - << commands_created << "/" << commands_deleted << endl;
  115 + << commands_deleted << "/" << commands_created << endl;
116 116 }
117 117  
118 118 exited = true;
... ... @@ -261,6 +261,7 @@ void Redox::process_queued_commands() {
261 261 else if(process_queued_command<int>(id)) {}
262 262 else if(process_queued_command<long long int>(id)) {}
263 263 else if(process_queued_command<nullptr_t>(id)) {}
  264 + else if(process_queued_command<vector<string>>(id)) {}
264 265 else throw runtime_error("[FATAL] Command pointer not found in any queue!");
265 266 }
266 267 }
... ... @@ -295,6 +296,9 @@ Redox::get_command_map&lt;long long int&gt;() {
295 296 template<> unordered_map<long, Command<nullptr_t>*>&
296 297 Redox::get_command_map<nullptr_t>() { return commands_null; }
297 298  
  299 +template<> unordered_map<long, Command<vector<string>>*>&
  300 +Redox::get_command_map<vector<string>>() { return commands_vector_string; }
  301 +
298 302 // ----------------------------
299 303 // Helpers
300 304 // ----------------------------
... ... @@ -314,7 +318,7 @@ string Redox::get(const string&amp; key) {
314 318  
315 319 auto c = command_blocking<char*>("GET " + key);
316 320 if(!c->ok()) {
317   - throw runtime_error("[FATAL] Error getting key " + key + ": " + to_string(c->status()));
  321 + throw runtime_error("[FATAL] Error getting key " + key + ": Status code " + to_string(c->status()));
318 322 }
319 323 string reply = c->reply();
320 324 c->free();
... ... @@ -322,7 +326,6 @@ string Redox::get(const string&amp; key) {
322 326 };
323 327  
324 328 bool Redox::set(const std::string& key, const std::string& value) {
325   -
326 329 return command_blocking("SET " + key + " " + value);
327 330 }
328 331  
... ...
src/redox.hpp
  View file @9e898af
... ... @@ -194,6 +194,7 @@ private:
194 194 std::unordered_map<long, Command<int>*> commands_int;
195 195 std::unordered_map<long, Command<long long int>*> commands_long_long_int;
196 196 std::unordered_map<long, Command<std::nullptr_t>*> commands_null;
  197 + std::unordered_map<long, Command<std::vector<std::string>>*> commands_vector_string;
197 198 std::mutex command_map_guard; // Guards access to all of the above
198 199  
199 200 // Return the correct map from the above, based on the template specialization
... ... @@ -272,7 +273,6 @@ Command&lt;ReplyT&gt;* Redox::command_blocking(const std::string&amp; cmd) {
272 273 Command<ReplyT>* c = command<ReplyT>(cmd,
273 274 [&val, &status, &m, &cv](const std::string& cmd_str, const ReplyT& reply) {
274 275 std::unique_lock<std::mutex> ul(m);
275   -
276 276 val = reply;
277 277 status = REDOX_OK;
278 278 ul.unlock();
... ...