Comments on: Multi-Model Benchmark: Round 1 Results | ArangoDB Blog

By: Alan Plum

Alan Plum — Tue, 26 Jan 2016 17:29:00 +0000

ArangoDB is a database system, so it is an alternative to MySQL, PostrgreSQL, MongoDB and so on, not something you install on top of them. What driver you use depends on what programming language you want to use to build your application.

By: Zack

Zack — Thu, 07 Jan 2016 08:19:00 +0000

Hi, I am really sold on the idea of arangodb and would like to use the database, however from my tests I have found arangodb aggregation to be slow. For example if you replace the count aggregation in the aggregation test with calculating the average or sum, arango compares slow to mongodb.

By: fceller

fceller — Tue, 21 Jul 2015 15:05:00 +0000

In reply to Dário Marcelino. @dmarcelino:disqus the select statement returns only the keys. We were asked to do a second test, where we return the whole profile documents (instead of just the keys).

By: Dário Marcelino

Dário Marcelino — Mon, 20 Jul 2015 10:12:00 +0000

In reply to Claudius Weinberger.

I see, I assume the SQL in question is:

> SELECT set(out_Relation.key, out_Relation.out.Relation._key) FROM Profile WHERE _key = :key

I don’t see anything wrong with it but I’m not an OrientDB SQL expert, I mainly use the Node.js driver. Hopefully someone more knowledgable will intervene.

Thanks

By: Claudius Weinberger

Claudius Weinberger — Sun, 19 Jul 2015 09:47:00 +0000

In reply to disqus_8NK4qKyTJQ. That depends on the database, the language the database is programmed, your infrastructure, architecture and more. So it's hard to answer your question.

By: CoDEmanX

CoDEmanX — Sat, 18 Jul 2015 14:22:00 +0000

Unbelievable how much they lie at Orient Technologies… They call the dataset dirty although there was just a problem with the import, they change benchmark conditions and don’t mention that they used caches (hello?! It needs to be computed at some point…), they pretend they improved their DB without acknowledging your benchmark that started the discussion and they say their new Node.js driver would do any better, although they did nothing but a name change. Honestly, I hope you consider anti-competition charges because of false advertising messages if they continue like that.

By: disqus_8NK4qKyTJQ

disqus_8NK4qKyTJQ — Thu, 16 Jul 2015 19:48:00 +0000

In comparing the drivers, in which order is the perfomance improved. Like, C++, nodejs, javascript, blueprint/gremlin, go and the other drivers. I’m interested in one of the most promising drivers.Thank you

By: Claudius Weinberger

Claudius Weinberger — Wed, 15 Jul 2015 14:32:00 +0000

In reply to Dário Marcelino. Yes, we will included some highly asked databases in the next round. And yes, we will extend the neighbors tests to include the documents as well with the next round of databases. We have not included it now, because we were waiting for an officially sanctioned OrientDB SQL to do it. We are not allowed to post in the OrientDB mailling list, but someone asked for a statement there. If we get an answer we will include OrientDB as well.

By: Dário Marcelino

Dário Marcelino — Wed, 15 Jul 2015 14:16:00 +0000

It’s great to see you guys are still updating the benchmark and even better you’ll add more DBs. As other people I would love to see how, for example, Postgres compares.

I have a question which has been a controversy point in the past: is the latest version of neighbours2 fetching whole documents?

BTW, even though I understand why beta and alpha versions are being compared here, and it’s interesting to see the progress made, using production versions is the definitive test since it’s what people actually use. Beta and alpha versions are useful in giving a glimpse into the future but they can lose some of their performance while maturing to production grade.

By: Ben

Ben — Wed, 15 Jul 2015 13:28:00 +0000

I really took the time and efforts to run the tests, thanks for open sourcing them and answering so quick to some additional questions I had around the set up – saved me a lot of time. Since multi model and especially the graph functionality are currently my requirements for a DB, I just focused on ArangoDB, Neo4J and OrientDB.

First some context info:
– I used the machines as in the description
– Even though I would never use alphas/betas in production, I went for ArangoDB 2.7 alpha, OrientDB 2.2 alpha and Neo4J Community 2.3.0-M02 (I could not get the Neo4J version used in the test)
– I ran the test 3 times per DB
– Memory usage is not my main concern and the set up for measuring was kind of time consuming, so I dropped it for my benchmark.

My results are quite in the range of this post. One striking difference was just on the aggregation by Neo4J (my used version) which was worse than in the test. The other results are similar and definitely within an acceptable and normal deviation of the results in the post. Here my results:

ArangoDB

shortest: 0.036
neighbors 2nd: 0.191
single read: 22.230
single write: 23.177
single write sync: 135.949
aggregation: 1.761

Neo4J

shortest: 0.474
neighbors 2nd: 3.170
single read: 165.320
single write:
single write sync: 227.201
aggregation: 12.407

OrientDB

shortest: 0.276
neighbors 2nd: 3.475
single read: 51.150
single write: 20.617
single write sync:
aggregation: 31.092