go-mysql-server is the SQL query execution engine that powers Dolt and DoltHub. Today we are excited to announce that we are adopting the project after its founding company ceased operations. Our fork of the project has over 400 additional commits that fix bugs, add new features, and greatly expand test coverage. We plan to continue developing and supporting the project indefinitely.

This blog discusses the history of Dolt's relationship with go-mysql-server and how our fork has diverged from the parent since we began developing it.

Switching from our janky SQL engine to go-mysql-server

Our first attempt at SQL execution for Dolt involved writing our own engine. Writing a SQL engine is very fun, but very hard. There's a lot of surface area to cover to be correct before you even start worrying about performance. Especially when you have no idea what you're doing and are basically winging it. Discovering the go-mysql-server project really saved Dolt's SQL capabilities at a critical time, dramatically increasing the feature set we were able to deliver for our beta. There were some bumps in our integration, and we found plenty of bugs and missing features, but on the balance there was really no question that switching to go-mysql-server was the right choice for Dolt.

We submitted our first PR to the parent repo back in July of 2019, shortly before Dolt's beta release. It was a bug fix for OFFSET and LIMIT clauses. We managed to merge back several more PRs before the project died around November 2019. We continued development on our fork up until today.

New features on our fork

Since the parent org went silent, we've developed many new features for the engine and fixed a lot of bugs we found. Here's a brief overview:

• INSERT, UPDATE, DELETE, and REPLACE statements (the engine was read-only when we started work on it)

• Most DDL statements, such as CREATE TABLE, DROP TABLE, ALTER TABLE, and many more.

• View support, including CREATE, DROP, and other DDL statements, as well as execution.

• Indexed join execution (limited to 2 tables for now)

• INSERT INTO SELECT statements, such as INSERT INTO foo SELECT * FROM bar;

• Windows support (project would not build on Windows before)

• Dramatic expansion in supported types:

  • DATETIME
  • TIMESTAMP
  • ENUM
  • SET
  • VARCHAR and VARBINARY
  • Full collation and character set support for all string types
  • Support for string length parameters

• DESCRIBE statements for tables and queries

• UNION queries

• AS OF queries

• Index DDL statements

• COMMIT, as well as support for AUTOCOMMIT session variable

• Lazy-loading indexes

• Better alias support

• Better case insensitivity (work in progress)

• Many more functions:

  • ABS()
  • TIMESTAMP()
  • DATETIME()
  • UNIX_TIMESTAMP()
  • USER()
  • LEFT()
  • INSTR()
  • IF()
  • SCHEMA()
  • CURRENT_USER()
  • RAND()

• Better support for INFORMATION_SCHEMA tables

• Test harness for sqllogictest

Conclusion

Dolt owes a lot of its success to the work done by the go-mysql-server project. We're very excited to take over the project and to continue improving it. We're grateful to all the developers that contributed to it so far, and welcome any contributions and issues from the community!

Download Dolt today to try out the capabilities of the engine, or try running some queries on the web.