The future of deepstream & JSON compatibility in CockroachDB

hello -

it would appear that a deepstream interface may not happen anytime soon, as per this reply. this is particularly surprising (and maybe a little disappointing) given cockroaches key-value foundation. i should think that deepstream and cockroach would be a natural fit for a future ‘full-stack’ framework.

cockroachDB seems perfectly capable of managing a JSON object for us, but i will be unable to take advantage of any cockroachDB features to access this stored deepstream JSON object.

while working with a deepstream–>cockroachDB interface, i am simultaneously working with one for MariaDB, which honestly is much further along in a deepstream interface.

i am certainly willing to wait, but it would be very nice to know that my efforts (and the money i spent on the interface) might someday prove useful.

2 Likes

Hi @edwardsmarkf, thank you for your commitment to our project! We hear you that JSON compatibility is important for you, and have marked this down in our roadmap. However, we have some other developer usability key priorities that we are hashing out first, including better support for ORMs. You can check out our ongoing roadmap here for Q4, and we will be sharing a Q1 roadmap as that approaches. https://github.com/cockroachdb/cockroach/issues/10528

@dianasaur323 I’m not sure how set in stone the priorities are, but so long as the topic is being discussed, JSON/proto column support would be a huge win for some of the use cases I have as well.

In particular it would be awesome if there were a Postgres-esque Generalized Inverted Index to do “automagic” indexing of sub-hierarchies within the column structure. Tho I’m aware that their implementation is heavily reliant on the B-Tree structure underlying PG, so it’ll be non-trivial to replicate in Cockroach :slight_smile:

Hi @twrobel, priorities are always shifting depending on feedback. I’m curious to hear what specific use cases you are thinking of for JSON?

Also, which is more important to you - ORM compability or JSON?

@dianasaur323 I can’t go into much detail, unfortunately, but our biggest use case where something like Postgres’ GIN implementation on jsonb types would be a huge win is in the case where a user provides some structured data themselves, and want to be able to search for it later on via the data they provided. We have some applications where this is a key requirement, but where the amount of data exceeds that which PG can reasonable handle before starting down the rabbit hole of sharding.

As for the ORM compatibility vs JSON, JSON is more important for my use cases. Due to the data scale I work with, most of our database queries are hand written and optimized anyways, or at most use a query builder such as Knex, so support for full-blown ORMs won’t have big impact for us. (With that said, compatibility with schema management and migration tools such as Flyway would also be great, from a administrative perspective).

I see JSON/structured column support as a core DB feature and ORMs as more of a developer niceity, which is why I lean towards favoring the former, tho I also understand that good ORM compatibility lowers the barrier of entry for most folks :slight_smile:

@twrobel I see, and the reason for wanting JSON in that scenario is that that data may not fit into a typical relational schema. Thank you for providing that use case.

Regarding why we are building ORM compatibility, your last comment is spot on. We worked on Hibernate this quarter, and next up on the docket is likely Sequelize. We have marked JSON support down in our roadmap, and given the feedback here, are considering moving it up in timing.

Thanks for bringing up Flyway - we are thinking about data migrations in the upcoming quarter, so this is good feedback on how to implement it.

1 Like

Hi,

We are developing a banking application and need JSON datatype as well as liquibase data migrations

i am very interested to learn how you are combining deepstream with cockroach. you might also consider looking at the sequelize interface as well.