# The eXtremeDB XML C API

> For C applications, the eXtremeDB schema compiler option “`–x`” causes `mcocomp` to generate interface functions to retrieve, create and replace (update) the contents of an object with the content of an XML string. Usage of the static and generated XML APIs are explained in the following sections.

> ## XML export and import

> The XML export and import functions are used in conjunction with user-defined file I/O helper functions to stream eXtremeDB database contents to and from persistent media files:

>         MCO_RET mco_db_xml_export(mco_trans_h t, void* stream_handle, mco_stream_write output_stream_writer);

>         MCO_RET mco_db_xml_import(mco_trans_h t, void* stream_handle, mco_stream_read input_stream_reader);

> When `mco_db_xml_export()` is called the internal runtime implementation calls the user-defined handler `output_stream_writer` to manage the output stream. And likewise `mco_db_xml_import()` causes the handler `input_stream_reader` to be called. Simple file I/O handlers look like the following:

>         mco_size_sig_t file_writer(void* stream_handle /* FILE*  */, const void* from, mco_size_t nbytes)
>         {
>             return (mco_size_t) fwrite(from, 1, nbytes, (FILE*) stream_handle);
>         }

>         mco_size_sig_t file_reader(void* stream_handle /* FILE*  */, void* to, mco_size_t max_nbytes)
>         {
>             return (mco_size_t) fread(to, 1, max_nbytes, (FILE*) stream_handle);
>         }

> Function `mco_db_xml_export()` may be called within a `READ_ONLY` transaction but, as expected, `mco_db_xml_import()` must be called within a `READ_WRITE` transaction. When importing data into an existing database, new data will be added into the database and no existing data will be lost.

> `mco_db_xml_import()` calls the internal function `mco_w_xml_create_object()` to create a new object in the database for each object loaded from the XML stream. Since the import procedure runs in the context of a single transaction, there are only two possible results: either the whole data set will be imported successfully or no data will be imported. For example, if an object being imported creates a duplicate for any unique index, the entire transaction will be rolled back.

> ## Unique identifiers oid, autoid and autooid

> It is necessary to explain how eXtremeDB handles unique identifiers, i.e. fields of the types `oid`, `autoid` and `autooid`. As explained above, the import procedure only creates new objects and never updates existing objects. So if a class contains an `oid` field, the `oid` value from the XML stream is used in the newly created object. Care must be taken to assure that imported `oid` values do not duplicate values in existing database objects.

> In the case of “automatic id” fields of type `autoid` and `autooid`, the import procedure behaves according to the current XML policy settings. The policy switches `ignore_autoid` and `ignore_autooid`, when set to value `MCO_YES`, cause the procedure to ignore values for fields of this type in the XML stream and `mco_w_xml_create_object()` will generate the id values for the newly created objects just as if they were created by calling the `classname_new()` function. The default setting of the XML policy is `MCO_YES` for both switches as the safest setting to preserve database integrity.

> ## XML Policy

> The XML policy structure defines behavior options of the XML interface, such as string / blob encoding, XML indentation, etc. It can be set by the application at compile time and also at runtime via the policy APIs. These APIs and available options are defined in the eXtremeDB header file `include/mcoxml.h`.

> The default policy can be retrieved by calling the following function:

>         void mco_xml_get_default_policy( /*OUT*/ mco_xml_policy_t * p);

> To get and set the current policy the following functions are used:

>         MCO_RET mco_xml_get_policy( /*IN*/ mco_trans_h t, /*OUT*/ mco_xml_policy_t * p);

>         MCO_RET mco_xml_set_policy( /*IN*/ mco_trans_h t, /*IN*/ const mco_xml_policy_t * p);

> To set the current policy requires a `READ_WRITE` transaction and the policy changes become effective immediately on calling `mco_xml_set_policy()`, not when the transaction is committed.

> For example:

>         void ChangeXMLOutput(void)
>         {
>             mco_xml_policy_t policy;
>             mco_trans_h   t ;

>             mco_trans_start(db,MCO_READ_WRITE,MCO_TRANS_FOREGROUND,&t);
>             mco_xml_get_policy(t, &policy);
>             policy.text_coding = MCO_TEXT_BASE64; //BASE64;
>             policy.blob_coding = MCO_TEXT_BASE64; //BASE64;
>             policy.ignore_field = MCO_YES;
>             mco_xml_set_policy(t, &policy);

>             // Do xml processing that depends on policy settings
>             ...
>             mco_trans_commit(t);
>         }

> Note on using alternative XML representation with attributes: Attributes are used to provide additional information about elements. In essence, XML elements can have attributes in the start tags, just like HTML. (See the web site http://www.w3.org/TR/2008/REC-xml-20081126/ for details.) To enable the XML representation using attributes, set the `use_xml_attrs` field of the `mco_xml_policy_t` structure to `MCO_YES`:

>         rc = mco_trans_start(db, MCO_READ_WRITE, MCO_TRANS_FOREGROUND, &t);
>         policy.use_xml_attrs = MCO_YES;
>         mco_xml_set_policy(t, &policy);
>         rc = mco_trans_commit(t);

> ## Generated XML Interfaces

> As with all eXtremeDB generated interface functions, the XML functions are generated based on the DDL schema definition. Consider the following schema fragment:

>         struct Country
>         {
>             char<3> c_code;
>             string  name;
>         };
>         struct Date
>         {
>             uint1   day;
>             char<3> month;
>             uint2   year;
>         };
>         struct Passport
>         {
>             char<8> series;
>             uint8    number;
>         };
>         struct Address
>         {
>             Country  country;
>             string   city;
>             string   street;
>         };
>         struct Phone
>         {
>             int2    country;
>             char<5> area;
>             char<7> number;
>         };
>         struct Residence
>         {
>             Address where;
>             Date    since;
>             optional Phone   phone;
>         };
>         struct Office
>         {
>             Address         where;
>             string          organization;
>             string          position;
>             vector<Phone>   phone;
>         };
>         class Person
>         {
>             string    name;
>             Residence residence[3];
>             optional Office    office;
>             optional Phone     mobile;
>             blob      description;

>             oid;
>             autoid[100];

>             list;
>         };

> ### _xml_get()

> For each class declared in the schema, the DDL compiler generates an `_xml_get()` function that is generally used (contrary to the implication of “get”) to write the XML representation of a database object to an output stream:

>         MCO_RET <classname>_xml_get( /*IN*/ classname *handle, /*IN*/ void * stream_handle, /*INOUT*/ mco_stream_write o_stream);

> The following code snippet demonstrates how the `_xml_get()` function would be called within a `READ_ONLY` transaction to write database objects in XML format to the output stream:

>         MCO_RET output_data (mco_db_h db)
>         {
>             mco_trans_h  t;
>             Person       p_obj;
>             MCO_RET      rc;
>             mco_cursor_t c;
>             FILE * f = fopen(xml_name, "w");

>             rc = mco_trans_start(db, MCO_READ_ONLY, MCO_TRANS_FOREGROUND, &t);
>             if(rc) return rc;

>             // instantiate a 'list' cursor for Person class
>             rc = Person_list_cursor(t, &c);
>             if(rc) return rc;

>             // iterate over the Person objects
>             rc = mco_cursor_first(t, &c);
>             if(rc) return rc;

>             for(;;)
>             {
>                 // get Person object handle from the cursor
>                 rc = Person_from_cursor ( t, &c, &p_obj);
>                 if(rc) return rc;

>                 // write Person object as XML through the 'doprint' function
>                 rc = Person_xml_get(&p_obj, f, &do_print);
>                 if(rc) return rc;

>                 // advance the cursor, break loop if end of list
>                 if ( mco_cursor_next(t, &c) != MCO_S_OK )
>                     break;
>             }
>             rc = mco_trans_commit(t);
>             return rc;
>         }

>         int do_print( /*IN*/ void *stream_handle, /*IN*/ const void * from, /*IN*/ unsigned nbytes)
>         {
>             // this simple example just writes the bytes to a FILE
>             // another example could write the bytes to a
>             // pipe to another process, a socket, etc.
>             FILE * f = (FILE*)stream_handle;
>             return fwritef( from, sizeof(char), nbytes, f );
>         }

> In this example, the stream is a file handle. It could have been a pipe to another process or any other type of stream. `Person_xml_get()` encodes the person object referenced by the handle, and calls the helper function `do_print()` passing the stream handle, a pointer to the XML string, and the length of the XML string.

> Note that the `classname_xml_get()` does not create any XML header. If an application is going to create a monolithic document with multiple XML objects, possibly of different XML tags, the application must create the appropriate XML header and footer entries to make it a legal XML document.

> ### _xml_put() and _xml_create()

> For each class declared in the schema, the DDL compiler also generates an `_xml_put()` function that is used to update existing database objects, and an `_xml_create()` function to add new database objects:

>         MCO_RET classname_xml_put( /*IN*/ classname *handle, /*IN*/ const char * xml );

>         MCO_RET  classname_xml_create( /*IN*/ mco_trans_h t, /*IN*/ const char * xml, /*OUT*/ classname * handle);

> The `_xml_put()` function updates an existing object from the XML description. The entire object is updated; it is not possible to selectively update fields. (Note that the class handle will have been established from, for example, a cursor, and carries the transaction context with it - hence, it is not necessary to pass a transaction handle to this function).

> The `_xml_create()` function creates an object from the XML description. The first parameter is the transaction context, the second parameter is the XML description and the output parameter is a handle to the new object. The following code snippet demonstrates how an XML input file could be parsed to obtain an object’s XML description, then call `_xml_create()` to insert the new object into the database:

>         static int insert(mco_db_h db, FILE *file)
>         {
>             MCO_RET     rc;
>             mco_trans_h t;
>             Person      p_obj;
>             int         c;

>             for (;;) /* loop on xml-objects */
>             {
>                 /* start the transaction */
>                 rc = mco_trans_start(db, MCO_READ_WRITE, MCO_TRANS_FOREGROUND, &t);
>                 if(rc) return rc;

>                 /* skip everything before a class tag */
>                 do c = getc(file);
>                 while ( c != '<'  &&  c != EOF )
>                     ;
>                 if ( c == EOF )
>                     break; /* end-of-file, thus finished */

>                 ptr = 1;
>                 xml[0] = '<';
>                 /* read the class name */
>                 do
>                 {
>                     c = getc(file);
>                     xml[ptr++] = c;
>                 } while ( c != '>'  &&  c != EOF );

>                 if ( c == EOF )
>                     break; /* finished */

>                 xml[ptr] = 0;
>                 if ( strcmp(xml, "<Person>") != 0 )
>                     exit(1);

>                 /* read xml-object */
>                 for (;;)
>                 {
>                     c = getc(file);
>                     if ( c == EOF )
>                     {
>                         xml[ptr] = 0;
>                         printf("\n Error - unexpected end of file: %s\n",
>                             &xml[(ptr>50)?ptr-50:0]);
>                         exit(4);
>                     }
>                     xml[ptr++] = c;
>                     if ( c == '>' )
>                     {
>                         xml[ptr] = 0;
>                         /* closing tag, the object is complete */
>                         if ( strcmp("</Person>", &xml[ptr-9]) == 0 )
>                             break;
>                     }
>                 }

>                 /* write database.. */
>                 rc = Person_xml_create(t, xml, &p_obj);
>                 if ( rc != MCO_S_OK )
>                     exit(0);

>                 rc = mco_trans_commit(t);

>                 /* ... and start over */
>                 ptr = 0;
>             } /* loop on xml-objects */

>             return rc;
>         }

> In the above example, an XML string is parsed to find the class tag. This example only deals with `Person` objects, so if class tag is for any other type of object, the procedure terminates. Otherwise, the XML is read up to the closing `Person` tag (“</Person>”). Then `Person_xml_create()` is called, passing a transaction handle, the XML string, and the handle of a `Person` object that will reference the newly created object.

> Note that if an XML document was created by eXtremeDB and contains just a single XML object, there is no need to parse the opening and closing tags; just read the entire XML into a buffer and pass it to `_xml_create()`.

> If the `Person` object represented in the XML string already existed in the database (i.e. an attempt was made to violate a unique key constraint), `Person_xml_create()` would fail with code `MCO_S_DUPLICATE`. If this is a possibility, the application should be written to find the key values within the XML string and attempt to locate the object first, and then call the `_xml_put()` or `_xml_create()` function accordingly. This process is illustrated in the following pseudo-code:

>         for (;;) /* loop on xml-objects */
>         {
>             /* start the transaction */
>             /* skip everything before a class tag */
>             /* read the class name */
>             if ( strcmp(xml, "<classname>") != 0 )
>                 exit(1);

>             /* read xml-object */
>             /* extract key fields */
>             /* write database.. */
>             if((rc = classname_fieldname_search(. . .)) == MCO_S_OK)
>                 rc = classname_xml_put(. . .);
>             else
>                 rc = classname_xml_create(. . .);
>             if ( rc != MCO_S_OK )
>                 exit(0);

>             rc = mco_trans_commit(t);
>             /* ... and start over */
>         } /* loop on xml-objects */

> ### _xml_schema()

> The last generated function for each class declared in the schema is the `_xml_schema()` function used to write the XML description of this class to the output stream.

>         MCO_RET classname_xml_schema( /*IN*/ mco_trans_h t, /*IN*/ void * stream_handle,
>                             /*INOUT*/ mco_stream_write o_stream );

> The `_xml_schema()` function must be called in the context of either a `READ_ONLY` or a `READ_WRITE` transaction. The output schema format is compliant with the W3C specifications, which can be found in the following documents: http://www.w3.org/TR/xmlschema-0 and http://www.w3.org/TR/xmlschema-1

> (The current implementation of this function only supports the default XML policy.)

> Example:

>         MCO_RET output_schema (mco_db_h db )
>         {
>             mco_trans_h t;
>             FILE * f = 0;
>             MCO_RET rc;

>             mco_trans_start(db, MCO_READ_ONLY, MCO_TRANS_FOREGROUND, &t);
>             f = fopen("person.xsd", "w");
>             Person_xml_schema( t, f, &do_print );
>             fclose(f);
>             rc = mco_trans_commit(t);

>             return rc;
>         }

> The XML schema can be used in conjunction with tools, such as XMLSpy, to validate the content of XML documents, which you can do prior to attempting to import the XML document into eXtremeDB.

> The XML document can also be used with XSLT, which is a language for transforming XML documents into other XML documents. This might be a necessary step to exchange data between eXtremeDB and an external system if they don’t have identical representations for the data being exchanged. For further information on XSLT, please refer to http://www.w3.org/TR/xslt20/

> ## Output Data Format

> Whether using `mco_db_xml_export()` or a generated `_xml_get()` function to export data, the XML output always starts with the <`classname`>. If the value of the indent field in the policy is 1, a new line is started and `indent_element` spaces are added. The `indent_element` constant is defined as two spaces. Every time it is necessary to step down one level in the hierarchy, this process is repeated. A new level is started when a structure, a vector or an array is processed.

> - `uint1, uint2, uint4, int1, int2, int4` are written as the appropriate integers (unsigned or signed). The base is defined in the policy field `int_base`, which could be 8, 10 (default) and 16.
> - `uint8, int8` are written similar to the other integers, but decimal format is not allowed.
> - `autoid` is formatted as `uint8`
> - `date, time` are formatted as `uint4`
> - output for `float` and `double` depend on the policy’s `float_format` field value. `MCO_FLOAT_FIXED` means `floats` are formatted as fixed-point numbers (eg. `0.0025`); `MCO_FLOAT_EXPONENT` means floats are represented in exponent form: i.e. integer part, fractional part and exponent (eg, `2.5e-3`)
> - `oid, ref` are coded into hexadecimal format
> - `blob` depends on the `blob_coding` value: `MCO_TEXT_ASCII` means ASCII as defined by the XML specifications, `MCO_TEXT_BINHEX` means BINHEX (2 hexadecimal digits / byte), `MCO_TEXT_BASE64` (Base64: this is a non-standard XML enhancement specific to eXtremeDB)
> - `char, string` formatted in accordance to the policy’s text_coding field: `MCO_TEXT_ASCII` means ASCII as defined by the XML specifications (default), `MCO_TEXT_BINHEX` means BINHEX, `MCO_TEXT_BASE64` (Base64: this is a non-standard XML enhancement specific to eXtremeDB)

> ## Input Data Format

> Whether using `mco_db_xml_import()` to import data or a generated `_xml_put()` or `_xml_create()` function to convert the XML description to eXtremeDB data format, the XML input will be converted according to the following rules:

> - `uint1, uint2, uint4, int1, int2, int4` are decoded from the incoming XML natively. The numeric base is determined based on the first characters; 0 for octal, 0x – for hex, otherwise the number is considered decimal. For example, `0643` – octal, `419` – decimal, `0x1A3` – hex; hexadecimal notation allows using capital or low-case letters (`0x1a3` is okay)
> - The number taken from the XML string is converted to the database type according to the dictionary. If the conversion is not possible without data loss, the appropriate error is emitted (for example, `MCO_E_XML_INVINT`, `MCO_E_XML_INTOVF`)
> - `uint8, int8` are decoded similar to other integers, but the base must be 8 or 16. If the base is 10 the error code `MCO_E_XML_INVBASE` is returned
> - `date, time` are considered `uint4`
> - `float, double` could be in exponential form or with the decimal point (i.e. `1.23e4` or `11.222`), otherwise the error `MCO_E_XML_INVFLT` is returned
> - `ref` should be written as hex value with 2 hexadecimal digits per byte
> - `blob` is decoded in accordance with the `blob_coding` value: `MCO_TEXT_ASCII` – ASCII as defined by the XML specifications, `MCO_TEXT_BINHEX` – BINHEX (2 hexadecimal digits / byte), `MCO_TEXT_BASE64` – Base64 (default: this is a non-standard XML enhancement specific to eXtremeDB)
> - `char, string` are decoded according to the text_coding value: `MCO_TEXT_ASCII` – ASCII (default), `MCO_TEXT_BINHEX` – BINHEX with two hex digits/per byte, `MCO_TEXT_BASE64` – Base64. If the resulting string length is longer than the size indicated by the dictionary, the string is truncated. If it is shorter, it is padded with zeros (‘\0’).