Here at DoltHub we believe in military and intelligence applications of Dolt. Harpoon Ventures, a venture capital firm that helps companies like ours get connected with the US military, invested in our last round of financing. In-Q-Tel is also an early investor. Harpoon helped the US Air Force become a Dolt customer via the SBIR program.

Today, inspired by a Podcast with Anduril CEO Brian Schimpf and this Defense Innovation Unit request, I'm going to discuss a very specific use case, Dolt for Drone Surveillance.

Drone Surveillance

For this use case you have a network of autonomous drones used to repeatedly survey an area with cameras and other sensors. The data from these sensors is summarized locally. Raw sensor data is too large to transfer en masse. The goal is to get summarized data back to human decision makers to enhance operational plans.

In this environment, a system must:

1. Limit Connectivity
2. Act Autonomously
3. Communicate Peer-to-Peer

Limit Connectivity

In this environment, communicating cellular or satellite data is dangerous. A cellular or satellite signal can be tracked and put your drone at risk. It is prudent for a drone to stay offline as much as possible. Moreover, communications can be denied by an enemy. Also, in remote areas, you may lack connectivity altogether. Thus, any solution for communicating data to crewed platforms or amongst other drones must be designed with limited connectivity in mind.

Act Autonomously

Because connectivity is limited, it is desirable that drones act autonomously based on local data. A drone network operates best with decentralized decision making. A drone must have enough information on board to make decisions about where to survey next without going online to get new orders.

Communicate Peer-to-Peer

In some situations, centralized connectivity via cellular or satellite may not be an option. In these situations, a viable alternative strategy is to use a peer-to-peer network to transfer data when in range of another drone. In this case, you must also have enough information on board to find a peer.

Dolt

How can Dolt help with this use case? Dolt is an offline-first, peer-to-peer database with fast, resilient sync. Dolt can form the database backbone of an efficient drone surveillance network.

In this use case, each drone and crewed platform is deployed with Dolt installed as its database to store operational data. Each drone or crewed platform acts as a Dolt remote to every other drone or crewed platform, forming a network of nodes. Communicating with other nodes can be done via a cellular or satellite network or peer-to-peer when necessary.

Dolt provides four novel capabilities for this use case:

1. Offline First
2. Fast, Resilient Sync
3. Conflict Detection
4. Tamper Resistance

Offline First

Each drone and crewed platform maintains a copy or clone of the operational databases on its local disk. In an offline or disconnected environment, the drones read and write from their local database.

The drone is free to read and write information to its local database knowing that it can sync that database with other nodes in the network later. No additional message queues are required to store changes for later syncing. The database maintains all the state necessary for sync.

I've used submarine drones in my illustrations to drive home the point that these drones are offline when submerged.

Fast, Resilient Sync

When a drone comes online, it synchronizes with any other drones or crewed platforms in the network. This network is resilient to one or many other offline nodes.

Writing to a local copy while still being able to sync writes to and from remote copies when connected is the novel Dolt capability. No other SQL database has this capability. When connectivity is restored, each drone’s Dolt database quickly computes the difference between data it has collected and other data in the network. Dolt then sends only the differences to other nodes in the network, other drones or crewed platforms.

This synchronization process is very efficient, effectively allowing the most data in and out in the shortest amount of time using the least amount of bandwidth. If communication is interrupted Dolt has the ability to resume where it left off.

Conflict Detection

In the event of conflicting writes, for example multiple drones produce different GPS locations of an object of interest, conflicts are surfaced to the application for further review.

In this case, the application can decide on the more trusted data with trust determined by other data in the database. These conflicts are stored in the version history and once the data makes it back to a crewed platform, the data can be human reviewed. These insights expose hard to find edge cases used to improve software or operational plans. Conflict detection and resolution in SQL is a novel Dolt capability.

Tamper Resistance

Dolt is tamper resistant. Each Dolt commit is signed with an immutable hash built from the data. If the data stored on disk does not match the hash stored in Dolt, the data is corrupt and Dolt will fail to read it. This adds and extra layer of security beyond standard hardening techniques.

In the case data cannot be read, Dolt can read from or write to a previous known good version.

Specifics

Now that we understand the theoretical benefits of using a Dolt database for this use case, let's dive a bit deeper into specifics.

Database Schema

I'm imagining a database with three core tables. I'll explain these tables in order.

1. Map
2. Objects
3. Orders

Map

The "map" table is keyed by latitude, longitude, and last scan time and has an object identifier as a value which is NULL if there are no objects detected. You could store latitude and longitude down to 5 decimal places giving a precision of approximately 4 ft. In this case, 10B rows would cover approximately a 75 mile area. Conservatively, at 25 bytes per row (two doubles, 8 bytes and two integers, 4 bytes), the map table is approximately 250GB. You can make the precision of the latitude and longitude wider to shrink the table or increase the area covered. This is by far our largest table.

Objects

There would be a second table that stores the "objects of interest" detected. Let's say you expect between 100 and 1,000 objects of interest in a 75 mile radius. This table would be rather wide, storing potentially images, associated bounding boxes, and features of the object like length, width, has propulsion, has antenna, bearing, speed, etc. The vast majority of storage here would be images. Let's budget 10MB per object to be safe. At 1,000 objects, this table is 10GB.

Orders

There would likely be a third table that stored operational plan. This is information where to survey next, other drone locations, etc. This would contain information about other drones in the network to make sure work is not being duplicated. It is also the place where you would house the peer-to-peer data. This would be quite small, less than 1Mb, assuming the drone network is in the dozens or hundreds.

Operations

So in total we are talking about a database of approximately 300GB. Let's budget an additional 300GB for the history Dolt needs to perform fast sync. A 600GB database is well within the capability of Dolt and commercial SSD drives.

Dolt would be responsible for syncing these three tables between drones and crewed platforms. The interesting thing about Dolt in this use case is not what happens when a drone can find a peer to sync with but what it does when it cannot. The drone must decide offline whether to continue to surveil or to move towards the last known location of a peer and try to sync again. This is where Dolt's offline capabilities will really shine. The drone can continue to operate autonomously in this situation, making a decision and then continuing to read and write from the database.

Most of the time when a peer comes online to sync, the large 10B row table will be mostly the same, with very few new objects found. Potentially, you will update the position of a few already known objects. This sync will be fast because not much has changed in the model. You will also sync new operational plan data about other drones in the network, also small and fast.

In the case a new object is detected, this data must be synced throughout the network. This will be relatively slower. However, syncing this data is required in any other non-Dolt solution. Dolt will be able to resume sync where it left off if communications are interrupted, a unique Dolt capability.

So, Dolt's model is faster in the case where very little changes, which would be expected in a surveillance use case.

Conclusion

As you can see, Dolt makes an ideal database to back a drone surveillance network. Dolt's novel capabilities enable drones to operate autonomously and only sync data that changed, limiting connectivity requirements. We are excited to explore this use case more deeply with potential customers. Interested? Come chat with us on our Discord or email me at tim@dolthub.com.