Add support for PropertyGraphs in import/export DDL#2003
Add support for PropertyGraphs in import/export DDL#2003darshan-sj merged 7 commits intoGoogleCloudPlatform:mainfrom
Conversation
|
@darshan-sj As we spoke earlier, here's the first patch for PropertyGraph support in import/export. I will send the subsequent as I get them completed. |
|
@asthamohta @shreyakhajanchi An fyi, you were auto-selected for review. I've been in conversation with @darshan-sj regarding this effort. But I don't seem to have the permissions to change the reviewer to him. |
05206b3 to
100c4e1
Compare
|
The build should be fixed now. Turned out to be a dependency issue as OpenJDK 11 doesn't come with javaFX anymore (which Pair depends on). I updated and removed it. |
100c4e1 to
a06acc9
Compare
v1/src/main/java/com/google/cloud/teleport/spanner/ddl/Ddl.java
Outdated
Show resolved
Hide resolved
v1/src/test/java/com/google/cloud/teleport/spanner/ddl/DdlTest.java
Outdated
Show resolved
Hide resolved
|
Applied mvn spotless. I don't seem to have the permissions to add the label. But I believe the label should be "improvement" for this. |
8b748a8 to
9905704
Compare
9905704 to
0ef44bb
Compare
|
@darshan-sj Added Graph INFORMATION SCHEMA patch which populates Ddl.PropertyGraph by parsing information schema. One more patch pending for Import/ExportTransform, which I will push shortly. |
9f3459e to
63b1961
Compare
|
@darshan-sj I've pushed the code for import/export of graphs as well. I tested it and was able to export a graph from one DB and import it into another. I also tried exporting multiple graphs at the same time and it works. As mentioned in our other chat, the only remaining issue is that the ImportFromAvroTest for graph fails with a cryptic error inside Beam, so I've commented it out for now, but will reenable it once we get to the bottom of that issue. For all other purposes, DataflowTemplates for graph works well. |
darshan-sj
left a comment
There was a problem hiding this comment.
Except for one [Must fix] comment others comments are nits. Feel free to address them in subsequent PRs.
Overall changes look good!
v1/src/main/java/com/google/cloud/teleport/spanner/AvroSchemaToDdlConverter.java
Show resolved
Hide resolved
v1/src/main/java/com/google/cloud/teleport/spanner/AvroSchemaToDdlConverter.java
Outdated
Show resolved
Hide resolved
v1/src/main/java/com/google/cloud/teleport/spanner/AvroSchemaToDdlConverter.java
Outdated
Show resolved
Hide resolved
v1/src/main/java/com/google/cloud/teleport/spanner/AvroSchemaToDdlConverter.java
Outdated
Show resolved
Hide resolved
v1/src/main/java/com/google/cloud/teleport/spanner/AvroSchemaToDdlConverter.java
Outdated
Show resolved
Hide resolved
v1/src/main/java/com/google/cloud/teleport/spanner/AvroSchemaToDdlConverter.java
Outdated
Show resolved
Hide resolved
v1/src/main/java/com/google/cloud/teleport/spanner/AvroSchemaToDdlConverter.java
Outdated
Show resolved
Hide resolved
v1/src/main/java/com/google/cloud/teleport/spanner/DdlToAvroSchemaConverter.java
Show resolved
Hide resolved
v1/src/main/java/com/google/cloud/teleport/spanner/ddl/GraphElementTable.java
Show resolved
Hide resolved
v1/src/test/java/com/google/cloud/teleport/spanner/ImportFromAvroTest.java
Outdated
Show resolved
Hide resolved
715d4ee to
29d3f8e
Compare
|
The IT failure is not related to the code changes in this PR. |
No description provided.