Skip to content

CategoricalData/hydra

Repository files navigation

Hydra

Welcome to the Hydra project! Hydra is a unique functional programming language based on the LambdaGraph data model. It explores an isomorphism between labeled hypergraphs and typed lambda calculus: in Hydra, programs are graphs, and graphs are programs. Hydra has the ability to translate its own kernel into several other languages, including Haskell (Hydra-Haskell), Java (Hydra-Java), and Python (Hydra-Python). All three implementations are complete and pass the common test suite. Hydra is used for data modeling, validation, and transforms at Microsoft, while its closed-source predecessor Dragon was used for data integration and graph construction at Uber. The language is now being developed for its own sake, with the intention of becoming an Apache Incubator project, integrating more directly with Apache TinkerPop, and branching out into additional concrete programming languages. Typical use cases include:

  • Graph construction. Hydra supports TinkerPop-style property graphs as well as RDF and SHACL, and has been used in combination with the ISO/IEC GQL standard. Hydra provides DSLs for defining schemas and mappings, as well as tools for validating schemas and data, and moving them seamlessly into and out of the graph formats.

  • Data integration. Hydra includes "coders" (encoders+decoders) for many data and schema languages which you can easily compose together to build data transform pipelines. Some of the currently supported languages and formats include Protobuf, Avro, JSON and YAML, RDF formats including N-Triples, GraphQL, LinkedIn's PDL Schema language, as well as simple tabular data (CSV/TSV). Hydra has been used extensively with (Delta) Parquet, although this support is not currently open source. There is also limited support for C Sharp and Scala (Hydra-Scala).

  • Computational graphs. Unusually among graph data models and query languages, Hydra has deep support for polymorphism, as well as embedding computational elements with a graph (sometimes called computational knowledge graphs). As we mentioned, programs are graphs and vice versa. See the KGC 2024 presentation Graphs, logics, and lambda calculus for examples.

One of the less exciting, but very important aspects of Hydra is its test suite, which guarantees parity across the supported programming languages. This ensures that each Hydra implementation behaves the same, and we think it will be very useful in heterogeneous environments like TinkerPop where we need exactly the same logic -- validation, query steps, user-defined functions, etc. -- to be manifested identically in more than one programming language.

If any of the above sounds interesting, feel free to ask questions or get involved via the LambdaGraph Discord server. We are preparing for the 1.0 release, with all three core implementations (Haskell, Java, Python) now complete. Near-term goals include integrating with Apache TinkerPop, expanding language support, and growing the community. There is a recent demo video you can check out, as well as a couple of earlier presentations here and here, and the original design document here.

Documentation

Releases

All Hydra implementations share a single version number. The current release is 0.14.0; see the CHANGELOG for details.

Implementation Package Install
Haskell hydra on Hackage cabal install hydra or add to package.yaml
Java hydra-java on Maven Central net.fortytwo.hydra:hydra-java:0.14.0
Java (extensions) hydra-ext on Maven Central net.fortytwo.hydra:hydra-ext:0.14.0
Python conda-forge (in progress) Coming soon

See the release process for how releases are built and published.

About

Graph programming language

Resources

License

Stars

Watchers

Forks

Packages