See our new articles on Medium – Capture Enterprise Metadata and Is InfinityDB a Document or Tabular DBMS? and our new slide deck.

InfinityDB Server meets and exceeds the flexibility of document databases without sacrificing critical relational features like joining, sorting, ACID transactions, immediate consistency, referential integrity, strong data typing, and a complete query language. Databases scale to at least terabytes, and performance scales with CPU cores. No complex expensive clustering is needed. Run on AWS marketplace or on premises.

InfinityDB Server is a complete secure user-friendly web-based DBMS. Software clients or users securely access dynamic custom REST APIs through PatternQueries to conceal the flexible evolving schema. For users there is no coding or command line interface. PatternQueries are like complete mini-applications but are concise, declarative, and readable, while operating on images, documents and nestable tabular structures with 12 data types. Edit anything in graphical, JSON, i-code, or CSV formats. Collaborate or backup on remote servers. Based on InfinityDB Embedded.We call the server v6.x.

InfinityDB Embedded (v4.x) and InfinityDB Encrypted (v5.x) are Java components for use in Java applications, licensable separately directly from Boiler Bay. They have superior performance, multi-core concurrency, compression, transaction support, safety and near indestructability. Authenticate and encrypt 100% of the data 100% of the time with AES256, HMAC-SHA256, X509 signatures and key rotation. The API and data model are very simple. Each db is a single file so administration is easy. Many custom uses. It is used in the server, but is inaccessible directly to application programs there in the same JVM. This has had thousands of users for decades. We call these v4.x and v5.x.

The Server

With the secure server, you effortlessly run queries or create and maintain data, schemas, REST Microservices APIs, queries, and indexing via the web to enhance productivity without the need for coding or configuration file editing, even for administration. Tuning, testing, exploration and experimentation use live edits to data and queries, without any server downtime or interruption.

PatternQueries

PatternQueries go far beyond SQL; they almost resemble programs but remain declarative rather than algorithmic. They are clear, concise, easy to read and learn, and editable as text or graphics. Each REST access URL corresponds to a single query, serving as a secure interface that conceals the database structure – nothing else is necessary. Queries are stored as structured data within their respective databases, travelling with the data and protecting it.

Universal Connectivity

Almost anything can easily be connected via tiny to elaborate APIs, as REST is a simple protocol supported by virtually all client languages. The server runs either on-premises or in a customer-launched Amazon AWS EC2 instance based on an AMI.

All Data Types

Images, pdfs or any other file or binary data (Blobs), documents, tables, tuples, sets, and lists may be flexibly nested, with 12 data element types. Metadata ‘class’ and ‘attribute’ data types are seamlessly embedded within the data, allowing for instant and incremental extensions. All data is more than just JSON documents as in other NoSQL DBMS; it is strongly typed, efficiently sorted, internally binary encoded for speed and compression, and interchangeable with any format on demand. The richness of the structures falls out from the expressiveness of the simple data model – spend 30 seconds to learn it. Blobs are simply broken down into smaller manageable units which then blend into the rest of the data.

Views

All of the data displays are just alternative views of a simple single underlying data model. Each view, data manipulation operation or data communication method applies to any data. Once the elementary data model is learned, all else falls in place.

Rich Display

Data exploration and editing are available in both graphical and text modes including JSON, CSV, and i-code. Graphically, data displays as a nestable mix of images, pdfs, or any other files or blob data, text, tables, documents and structures. Data is rich, flexible, informal or formalized, yet intuitive. PatternQueries can provide further aspects by transforming any data structure. See the Server for more details. Here are some views of data using the data browser in different formats over different data:

PaternQueries

The PatternQuery ‘language’ outperforms algorithmic languages such as Python, Java, or JavaScript while doing database access even though it is far simpler because it takes maximal advantage of the special characteristics of the underlying InfinityDB Embedded database ‘engine’.

A Tidy App

A PatternQuery is like a tidy little server-side application, while with SQL, there must also be complex, fixed, ugly algorithmic ‘glue’ code in the server to adapt to particular uses. The glue typically concatenates strings into SQL statements, executes, interprets result rows, and repeats this to assemble the output – an inefficient, error-prone, insecure, tedious mess. In contrast, a PatternQuery user or client application sees one formalized request and response. Requests and responses may be tiny or elaborate, small or large, either efficient binary blobs or JSON. They are immune to problems like SQL injection attacks.

Simple Richness

A PatternQuery effectively bundles any mixture of kinds of database operations, DML, DDL, and computation. Features include reducers, select, project, join, pattern matching, expressions, sorting, set operations, incremental schema evolution, indexing or ‘replication’, tree-like structure nesting, and much more. However, this richness is not a complication from the query author’s perspective because these are mostly available as a consequence of the simplicity of the data model, and there is no need to memorize specific language features for these concepts. A query is a pattern tree of literals expressions and symbols and a result tree of literals expressions and symbols, where the symbols can have descriptive attributes.

Optimized

The PatternQuery compiler is highly optimized, and while it offers optional fanout hints, it always completes the compilation phase within milliseconds, unlike traditional RDBMS systems. Query authors can customize queries to effectively embed optimal execution plans, such as the choice of search paths and joins of multiple replications or inversions. A unique ‘zig-zag’ algorithm significantly accelerates joins, intersections, and differences, all without requiring temporary storage, even in the absence of inversions. The pattern tree determines an efficient navigation of the inherently sorted data rather than a filtered scan. The underlying InfinityDB Embedded ‘database engine’ is highly optimized as well, and is multi-core threaded for speed and isolation of short and long operations.

Secure

Unlike hand-crafted REST interfaces and web servers, InfinityDB is secured by SSL/TLS for all communications. Administrative data is encrypted and passwords hashed. The admin creates users, roles, databases, permissions and grants, and the admin sets passwords. PatternQuery execute permissions are finely controllable via named interfaces, with queries optionally declared as ‘setters’, ‘getters’ or both. Permissions can be selective on these designations and on interface names or prefixes. Virtually all security and database administration are administered via the web, so there are no configuration files or server downtime, and management is simple and clear.

Scaleable

Symmetry

There is no fixed separation between ‘key’ and ‘value’ as in RDBMS, distributed hash tables or ‘document’ databases. Instead, all data is kept sorted and is randomly or sequentially searchable quickly, at all scales of granularity at all times, based on arbitrary prefixes of any length. A search prefix is like a variable-length composite key. All operations, queries, display, editing, and so on are symmetrical and scale-independent, so there is no special handling of 10 items or 10^12 Items for individual prefix-based retrievals, which take from a microsecond to a few milliseconds, even for updates. Hence it is useful to use the nesting capability to allow access with prefixes of various lengths and patterns at different times. This can be augmented with the unique ‘zigzag’ PatternQuery algorithm (see the examples).

Replications and Inversions

Large databases can benefit from ‘replications’ or ‘inversions’ of data in which different prefix structures can be used to keep access fast and avoid long sequential scans. PatternQueries explicitly maintain such replications, providing an opportunity for the author to do custom tuning and to get optimal performance. A replication is often just a permutation of the variable parts of multiple data Items, and an inversion – like an RDBMS index – is just a subcase with two variable parts. Novel replication architectures can be invented for dedicated purposes, such as an actual genome database that provides retrieval of genes based on ranges of DNA positions out of billions. Beyond simple permutations, replications can also include computations.

Remote

Remote servers and InfinityDB Embedded Java clients can be connected via a binary ‘ItemPacket’ protocol for consolidation, backup, curation, and collaborative purposes. The Server can be configured to use this to make databases that appear to be local actually be remote transparently to users and client software.

Custom architectures using this facility can be used in a wide range of distributed systems. The interface is simple, because it just implements the basic ‘ItemSpace‘ data model and API with almost no special code.

For more see InfinityDB Server where we show the web user interface and explain the data model and PatternQueries.

Applications

• Data Science using the server’s text or graphical database browser and PatternQueries
• Collaborative work involving people sharing both local and remote distributed data including images, files, tables, and nested data
• IoT or Microservices networks benefiting from PatternQueries defining the REST APIs
• Distributed database architectures enabled by the server’s ItemPacket protocol
• High performance Java applications utilizing the low-level ItemSpace and Map APIs with encryption.

Server Architecture

The InfinityDB Server version utilizes the architecture depicted below. It is designed for seamless deployment on Amazon Elastic Computing Cloud (EC2) via the AWS marketplace, allowing for instant launch as an ‘instance’, i.e. a virtual machine. Additionally, it can be executed on any platform where Java is supported with an ‘on-premises’ license. On the right-hand side, you can see the REST clients, which can encompass various devices such as computers, laptops, desktops, mobile devices, or IoT devices. The REST protocol is widely adopted and extremely user-friendly, making it accessible for any client program. A REST request functions like a remote procedure call, with each API method being handled by a straightforward declarative PatternQuery.

The remote servers represent other InfinityDB instances that have been configured as remotes within the referring server. From the perspective of clients, these remotes appear as regular databases. On the left side, you’ll find the web console, which serves as the interface for administration and interactive data access and querying. All communication is robustly secured through SSL/TLS (Secure Sockets Layer) encryption.

Within the EC2 instance, there are two Elastic Block Store (EBS) volumes: one serves as the root volume for the code, while the other is designated for data storage. The root volume is periodically updated to accommodate newer versions. When operating outside the EC2 environment, the use of two separate volumes is not required.

For more see InfinityDB Server. For quick usage instructions see web user interface. The InfinityDB Data Model is like a flexible document model with many extensions.

InfinityDB Embedded: The Underlying Database ‘Engine’

InfinityDB Embedded and InfinityDB Encrypted are optionally separately available from the server, and they comprise the ‘engine’ that underlies InfinityDB Server to provide the highest available performance, according to our customers and the provided performance tests. Many applications use these for custom purposes because of the flexibility and performance.

Performance

• More than 1M ops/sec are typical for multi-threaded insert, delete, and next in cache
• Multi-core overlapping operations scale almost linearly in thread count
• Almost all cores are used with many threads
• Threads use fair scheduling, with very low inter-thread interference
• Random I/O scales logarithmically in file size, with no size limit
• Huge caches are efficient – 1MB to 100GB or more, and are on-heap
• Transactions are fast: 50/s on disk, 300/s on flash, or thousands/sec for delayed durability
• Database open is immediate, even for recovery after abrupt exit

Features

 Here are the basic InfinityDB Embedded and InfinityDB Encrypted ‘Database Engine’ features. InfinityDB Encrypted is identical to InfinityDB Embedded but encrypts 100% of the database 100% of the time.

• Each standalone database is in a single file, used by a single JVM. The file format never changes.
• 10 simple API calls (insert, delete, delete suffixes, update, first, next, last, previous, commit, rollback)
• 12 basic data types:
  • String, double, float, long, boolean, date, byte array, byte string, char array, index, ‘Class’, ‘Attribute’
• Unlimited nestable data structures and patterns:
  • images, files, mime-typed BLOBs, trees, documents, inversions, lists, maps, sets, variable strongly-typed tuples, elegant Entity-Attribute-Value
• Schema is dynamically extensible, like an extended flexible JSON document model
• Data compression up to 10x or more, continuous, no memory or storage leaks
• APIs: fast ‘ItemSpace’, fast extended Java -standard ConcurrentNavigableMap adapter
• Zero administration
• High reliability, robust file update patterns avoid file systems’ inherent weaknesses
• Scalable to terabytes
• Scalable to dozens of cores

Sample Code

See the short embedded example code , map access example code, encrypted example code or client/server example code. Also see PatternQuery examples. A dzone.com article shows definitive performance testing with the Java Microbenchmarking Harness.

More Information

A full discussion is at InfinityDB Embedded and InfinityDB Encrypted. For a Free InfinityDB Embedded Trial Download see the shop. Here is the InfinityDB Embedded_Trial License.

For licensing, email support@boilerbay.com

The AirConcurrentMap Java ConcurrentNavigableMap

A separate product, the fast com.infinitydb.map.AirConcurrentMap API is identical to the Standard Java Maps – in fact it is a java.util.concurrent.ConcurrentNavigableMap, optimized for more than about 1K Entries. You can use our free non-commercial edition or license the commercial edition. Compare the performance with that of the Standard Java Maps. Memory efficiency is higher than any JDK Map as well. Our extensions provide extreme performance for parallel operations even beyond streams. The Map is fast and concurrent due to the same patented technology as is used in InfinityDB.

Learn more about AirConcurrentMap.

• AirConcurrentMap Free Non-Commercial Edition in the Shop
• Development and Deployment License