CockroachDB Beta Release 2016-10-13

What's new in CockroachDB beta-20161013


Hi there,

We've packed so much into this week's beta, including many features that have been in development since our stability-focused “code yellow” started back in August. When we entered “code yellow”, all work unrelated to stability was done in a secondary development environment. Since then, we’ve made great progress, and so have moved many of these new features back into our main development environment.

You can download the new beta here.


The Cockroach Labs Team


Backwards-Incompatible Changes

  • TIMESTAMP values are now stored with microsecond precision instead of nanoseconds. All nanosecond-related functions have been removed. An existing table t with nanosecond timestamps in column s can round them to microseconds with UPDATE t SET s = s + '0s'. However, note that this could potentially cause uniqueness problems if the timestamp is a primary key. #8864
  • TIMESTAMP values (without TIME ZONE) are now parsed in UTC, not in the session time zone. #9444
  • The deprecated --http-addr flag has been removed. When starting a node, use the --http-host flag instead to specify the address to listen on for Admin UI HTTP requests. #9725

SQL Language Changes

  • Tables can now be interleaved into other tables. Interleaving tables improves query performance by optimizing the key-value structure of closely related tables, attempting to keep data on the same key-value range if it’s likely to be read and written together. #7985
  • The CREATE TABLE AS statement can now be used to create a table based on the results of a SELECT statement. #8802, #9278
  • The ALTER TABLE ... SPLIT AT command can now be used to force a range split at a specified key. #8938
  • Added new information_schema metatables as well as initial support for the pg_catalog database. This work is part of our ongoing effort to provided standard database introspection required by popular ORMs. #8433, #8498, #9565, #9114, #9104
  • CHECK constraints can now be added with the ALTER TABLE ADD CHECK and ALTER TABLE VALIDATE statements. #9127 #9152
  • The ALTER TABLE DROP CONSTRAINT statement can now drop CHECK and FOREIGN KEY constraints. #8747
  • The SHOW CONSTRAINTS statement now requires the user to have privileges for the requested table. #8658
  • The EXPLAIN statement now works correctly for VALUES statements containing subqueries. #8970
  • The TRUNCATE statement now implements the CASCADE modifier. #9240
  • TIMESTAMP values that include a time zone can now omit the minutes field of the time zone offset, for compatibility with PostgreSQL. #8666
  • INTERVAL values can now be input in a colon-delimited format (H:M or H:M_S), for compatibility with PostgreSQL. #8603
  • The INT8 type is now supported as an alias for INT. #8858
  • INTERVAL, TIMESTAMP, TIMESTAMPTZ, and DATE values can now be casted to/from more types. #9731
  • The window functions row_number(), rank(), dense_rank(), percent_rank(), cume_dist(), ntile(), lead(), lag(), first_value(), last_value(), nth_value() are now supported. #8928, #9321, #9335
  • The exp() function returns an error if its argument is greater than 1024 instead of performing excessively expensive computation. #8822
  • The round() function now breaks ties by rounding to the nearest even value (also known as bankers’ rounding). It is now faster, and returns an error when given an excessively large number of digits. #8822
  • The concat_agg aggregate function is now supported. #9690
  • New date/time functions, extract_duration(), experimental_strptime() and experimental_strftime(), are now supported. #9734, #9762

Protocol Changes

  • The COPY protocol is now supported on the server side with the COPY FROM statement. The cockroach sql command-line tool does not yet recognize this command but it can be used with other client interfaces. #8756
  • DECIMAL values are now encoded correctly for drivers that use the binary format. #8319
  • TIMESTAMP, TIMESTAMPTZ and DATE values can now use the binary protocol format for clients that support it. #8590, #8762
  • NULL values are now sent with the correct type OID, which is required by some PostgreSQL drivers. #9331
  • The CREATE TABLE and CREATE DATABASE statements now return the standard PostgreSQL error code when the table or database already exists. #9235

Command-line Interface Changes

  • The cockroach start command now prints additional fields to stdout: clusterID, nodeID, and status, which indicates whether the node started a new cluster, joined an existing cluster for the first time, or rejoined an existing cluster. #9066
  • The cockroach start command now terminates with a non-zero exit status if it was interrupted with Ctrl+C or a signal. #9051
  • The cockroach zone set command now supports the --disable-replication flag, which sets the desired replication factor to 1. #9253
  • The cockroach gen example-data command can now be used to generate SQL for example databases. #9231, #9475
  • The cockroach sql shell now presents a new prompt when Ctrl+C is pressed after the user starts entering a statement. #9704
  • When using Ctrl+R in the cockroach sql shell to recall a previous statement from the shell history, the search is now case-insensitive. #9704
  • The cockroach sql shell now reports unterminated final statements as errors instead of ignoring them. #8838
  • The cockroach sql shell now reports when the connection to the server is lost and a new connection is opened. #9613
  • The cockroach sql shell now ignores lines that contain only whitespace (e.g., comments). #9243
  • The cockroach sql shell now prints expressions in EXPLAIN and the shell history such as E::T or E:::T in the same format entered instead of using the longer CAST or TYPE_ANNOTATE syntax. #9739
  • The cockroach sql shell now stops processing statements at the first error encountered when reading from a file (non-interactively). This behavior is customizable with the new \unset errexit shell command. #9610
  • The cockroach sql shell now checks if a statement or query is syntactically valid on the client side before it is sent to the server. This ensures that a typo or mistake during user entry does not inconveniently abort an ongoing transaction previously started interactively. This behavior is customizable with the new \unset check_syntax shell command. #9610
  • The cockroach sql shell now works better with multi-line statements. In particular, users will find it easier to recall and edit previously entered multi-line statements. This behavior is customizable with a new command \unset normalize_history. #9610
  • The cockroach sql shell now works correctly under kubectl and other environments that hide the size of the terminal. #9097

UI Changes

  • The time scale selector in the UI now works more reliably. #8573
  • Additional database and table-level details are now available. #9621
  • Displaying graphs for a longer timescales is now much faster. #8805
  • The default graph time scale is now chosen based on the age of the cluster. #9340
  • Node logs are now accessible in the UI. #8572
  • The UI can now be built with live reload support. #8679
  • Exposed idiomatic bucketed histograms on prometheus endpoint. #9810
  • Exposed Mutex timing metrics. #9769

Bug Fixes

  • Fixed some panics in handling invalid SQL statements. #9049, #9050
  • Fixed an issue that sometimes caused transactions to restart twice in a row. #8596
  • STRING literals containing non-UTF-8 data are now rejected. #9094
  • The UPSERT statement now works correctly during online schema changes.#9481
  • Aggregate and window functions in GROUP BY ordinal expressions are now rejected without crashing. #9629
  • The unsupported ALTER TABLE ... ALTER COLUMN ... SET TYPE ... statement now reports an error to the client instead of crashing the server. #8747
  • Slightly improved support for 32-bit systems. Note that 32-bit architectures remain officially unsupported. #9491
  • The admin UI now propagates timeouts to the server to avoid dangling requests. #8546
  • Fixed issues with duplicate column qualifications in CREATE TABLE statements #9868
  • Empty statements are now handled correctly in prepared statements. #9811
  • Fixed a Raft assertion caused by insufficient locking. #9814

Performance Improvements

  • Tuned RocksDB settings to reduce write stalls. #9663
  • Minor performance optimization to avoid reading an internal metadata key that will be immediately overwritten. #9263
  • Improved concurrency to prevent certain slow operations from dramatically impacting overall performance. #9622
  • DROP INDEX and DROP TABLE are now performed in chunks instead of one big transaction. #8870, #8885
  • The SQL query processor now tracks its memory usage. Soon it will return errors for queries that use too much memory. #8691
  • Aggregate functions like sum() are now faster. #8680
  • For production deployments, at least 10000 file descriptors are now recommended per store, with a minimum of 2000 required. #9679
  • Improved scheduling of Raft work. #9831
  • Replaced per-request checks with a per-batch check. #9848
  • Micro-optimizations for writing time series data. #9862
  • Narrowed the use of a lock during Raft log truncation. #9840
  • Refactored a common locking pattern for better performance. #9771

Doc Updates


This release includes 338 merged PRs by 28 authors. We would like to thank the following contributors from the CockroachDB community, especially first-time contributors Francis Bergin and Yan Long:

  • Francis Bergin
  • Daniel Theophanes
  • Yan Long
  • songhao