XML Deep Dive with the Magic xpa Application Platform

Photo by Ed Bierman under CC license. Some rights reserved.

As we mentioned in our last post on XML Programming Basics with the Magic XPA Application Platform, defining an XML data source is done by loading definitions from an XML schema (XSD). We begin our deep dive into XML on the Magic xpa Application Platform by discussing the default mappings used when converting from primitive .xsd data types.

As a review, when working with XML, information can be contained in either elements or attributes. There are no real rules about when to use elements or attributes. The same information can be stored as either an element or an attribute. In general, however, it is usually best to use attributes for metadata and nested child elements for data.

In the following two examples, the same information is presented:

<customer loyaltytype="Platinum">
  <firstname>Fred</firstname>
  <lastname>Smith</lastname>
</customer>

<customer>
  <loyaltytype>Platinum</loyaltytype>
  <firstname>Fred</firstname>
  <lastname>Smith</lastname>
</customer>

The Default Data Types table contains the XML default mapping, including all of the XSD types and their translation to Magic xpa attributes and pictures.

During a Get Definition, if an element’s or attribute’s type is defined using a SimpleType or as a reference to a Global element, it will be converted to the appropriate Primitive XSD data type from which it is derived. Primitive XSD data types will be mapped to corresponding Magic xpa attributes. 

As a review, a picture in Magic xpa is a string of characters that tells Magic xpa how to define the format of a Magic xpa attribute. You can specify the picture format in the Picture dialog box, which is accessed from the Picture property in the Details section of the Field Properties sheet. Magic xpa automatically defines the most commonly used pictures for data items that have known attributes (for example, dates). Magic xpa picture information could include information such as in this example:

Parameter Name	Attribute	Picture	Meaning
Company	Alpha	35	A 35-character alphanumeric parameter
Customer Number	Numeric	##### or 5	A 5-digit integer numeric parameter
Order Date	Date	MM/DD/YY	Date parameter format for input and output
Price	Numeric	######.## or 6.2	A number parameter with 6 whole digits and up to 2 decimals.

When dealing with XML, a default picture will be assigned as specified in the default data types table. When the type is based on a SimpleType, which defines length, the the Magic xpa picture will be updated with the length specified for the SimpleType.

You can edit the attribute and picture assigned to each data type. Each attribute/picture that differs from Magic xpa default attribute/picture is marked in green. These values will be saved in the [XML_DEFAULT_DATATYPES] section of the Magic.ini file.

You should be aware of the data mapping rules between XSD data types and Magic xpa data types:

1.  Mapping Repeatable Elements (MaxOccurs)

Multi-Occurrence elements in an XML document are handled the same way that “child” records are handled in ISAM or SQL data sources. The child records are displayed in a subtask, in a child form or subform, and are displayed using a range so only the records that relate to the parent are shown. Simple elements with maxOccurs > 1 (including “unbounded”) are considered to be compounds with one simple element, even if they are not compound, since they cannot be spread horizontally as with bounded simple elements. Each vector, even if simple, will never start in the record of its parent.

2.  Mapping Rules for anyType Elements

The XML Schema type xsd:anyType is the root type for all XML Schema types. All of the primitives are derivatives of this type, as are all user defined complex types. As a result, elements defined as being of xsd:anyType can contain data in the form of any of the XML Schema primitives as well as any complex type defined in a schema document. You use the xsd:anyType type as you would any other XML Schema complex type. It can be used as the value of an element element's type element. It can also be used as the base type from which other types are defined. The anyType elements are mapped in Magic xpa as BLOB fields. When loading an XML, the content of the element is loaded into the view. When writing the XML, any value modified in that field are written as-is to the XML.

3.  Mapping of HexBinary and Base64Binary

hexBinary [Definition:]   hexBinary represents arbitrary hex-encoded binary data. The ·value space· of hexBinary is the set of finite-length sequences of binary octets.

base64Binary [Definition:]   base64Binary represents Base64-encoded arbitrary binary data. The ·value space· of base64Binary is the set of finite-length sequences of binary octets. For base64Binary data the entire binary stream is encoded using the Base64 Alphabet in [RFC 2045].

The lexical forms of base64Binary values are limited to the 65 characters of the Base64 Alphabet defined in [RFC 2045], i.e., a-z, A-Z, 0-9, the plus sign (+), the forward slash (/) and the equal sign (=), together with the characters defined in [XML 1.0 (Second Edition)] as white space. No other characters are allowed.

In Magic xpa, HexBinary and Base64 Binary are automatically converted from/to binary content, when loading/serializing the XML into/from the view.

4.  Mapping of Date and Time Data Types

The XSD type date is read and written to an XML using the XML schema’s format YYYY-MM-DD, regardless of the ‘display picture’ selected for the field (in the ‘xml defaults’ or in the Studio).

Likewise, the XSD type time is read and written to the XML using the XML schema’s format HH:MM:SS. To access the optional milliseconds and time zone, developers need to change the attribute to ‘Alpha’, in the Studio, before loading the XML.

The types gYear, gDay and gMonth, each consist of a single numeric value and automatically support Numeric attributes of Magic xpa in the Studio.  When loading an XML, the numeric value is parsed from the XML into the flat-view (e.g., for a gMonth’s value ‘--10--‘, the flat-view field will contain the numeric value 10).  When modifying a value in the ‘flat-view’, the numeric value is formatted to the XML according to the format of the schema (e.g. 11 ->  ‘--11--‘).

All date and time types (including date, time, gYear, gDay & gMonth) allow Alpha attributes of Magic xpa, in which case the flat-view field will contain the entire content (e.g. ‘--10--‘ for gMonth). 

5.  Creation of Complex Elements

A complex element is an XML element that contains other elements and/or attributes. There are four kinds of complex elements: empty elements; elements that contain only other elements; elements that contain only text; elements that contain both other elements and text. Complex elements may contain attributes as well.

When creating a record in an XML data source using Magic xpa, a complex element that has no attributes or simple elements (and is part of a complex type of 'All' or 'Sequence') will automatically be created. This removes the need of defining an additional XML view for that complex element.

Creating, Modifying and Deleting an XML Record

When the main file is XML, all of the initial task modes are allowed. An inserted record may contain, in any sequence, fields of an existing parent record, or the Parent-ID column.

If you try and refer to a parent that does not exist, a new parent will not be created, but rather a message will appear stating that this is a "non-existing Parent ID".

In order to work in Create Mode you first have to select the Parent either explicitly by updating the ParentID variable or if not explicitly then implicitly  according to the modified elements. The runtime engine will try to decide to which parent the inserted record belongs to, according to the modified fields in the inserted record that belong to ancestor compounds. The locating algorithm will use all modified fields belonging to compounds above the view, and will try to fetch the parent record using these values.

Then in Create Mode you will need to execute the insert. Based on the Parent-ID that was explicitly or implicitly selected and validated in the previous phase Magic xpa will insert the record under the Parent record at the end of the view. You should note that Magic xpa will not validate XMLs implicitly. For example, ‘A.E’ or ‘A.D’ might have a restricting maxOccurs attribute. To ensure validity, you should call  the XMLValidate function.

The View Main Element controls which simple elements can be modified. Only simple elements and attributes contained within the View Main Element can be modified in the Record Suffix. Therefore, simple elements and attributes of ancestors are not modifiable. This means that relocations are not allowed. To move an element, you need to erase it in one place and then enter it in another place. In fact, developers should not modify the Parent-ID of an existing record as this will generate an error.

Deleting a compound’s record will delete records of its descendants.

XML Validation

XML validation is the process of checking a document written in XML to confirm that it is both well-formed and also "valid" in that it follows a defined structure. A well-formed document follows the basic syntactic rules of XML, which are the same for all XML documents. Magic xpa's XML Integration feature provides two functions that allow you to verify that the XML is compatible with the XML schema. These functions are XMLValidate which validates an XML document against its schema and XMLValidationError which returns the errors of the last XMLValidate.

XML Programming Basics with the Magic XPA Application Platform

XML is so commonplace today, that we may tend to forget what it really is and what it really does or does not do. First off, XML stands for Extensible Markup Language.  XML uses a flexible standards-based format, which enables you to easily share data. Unlike HTML whose purpose is to display data, XML’s purpose is simply to provide a format by which one can store and transport data. Unlike HTML, there are no predefined tags.

I made up my own tags for this example:

<blogger>
  <first_name>Glenn</first_name>
  <last_name>Johnson</last_name>

  <blog_name>Magic Application Platform</blog_name>

  <url>www.magicapplicationplatform.blogspot.com</url>
</blogger>

XML doesn’t do anything by itself, it is simply formatted data. Someone must write software to send, receive or display  XML. That’s where the Magic xpa Application Platform comes into the picture.

Magic xpa provides you with the ability to use XML documents as a standard data source in your applications. You can define an XML data source (XML view) in the Data repository and define a task to manipulate and handle the XML data.  

When using an XML data source, you can create Magic xpa tasks (Batch, Rich Client and Online) that read and manipulate XML documents using XML views. The XML views can be defined in tasks either as a main source or as a linked source.

You can also verify the validity of XML documents using the XMLValidate and XMLValidationError functions. XML schema namespaces are automatically handled by Magic xpa. XML and XSD files that use namespaces do not require any applicative translation between aliases and URIs.  

Earlier versions provided functions for working with XML, but XML manipulation is even easier with Magic xpa today. Now you can work with XML using the XML data source.

The following are provided by Magic xpa to enable you to work with XML data sources:

1. The XML File entry in the DBMS repository
2. The XML Defaults tab in the DBMS Properties dialog box
3. The Temporary Data Source property in the Database Properties dialog box
4. The XSD section in the Column Properties sheet
    

Defining an XML data source is done by loading definitions from an XML schema (XSD). If you do not  have a schema that describes the XML document structure that you want to work with, you should use an external tool, such as XMLSpy®, to generate the XML schema.

When parked on a line in the Data repository, you can define a single view for a specific XML schema.

When parked on top of the Data repository (# column), this utility will generate all required views for the XML schema and optionally will generate programs for reading and writing XML documents that are based on this schema. This method lets us reload schemas in order to update existing views.

If the schema file that you selected has several global elements (roots), the Root Selection list will appear. Select one of the roots in order to load it. For schemas with defined namespaces, the namespace URI will be displayed next to the global element name.

When you get the definition of an XML schema, and the Name column has no value, the column will be assigned with a default value: <schema file name with the suffix>.<root element name>. For example: If you use a file called "po.xsd", the name that will appear in the Name column will be "po.purchaseOrder".

Similarly, if the Data source name column has no value, the column will be assigned with a default value: <root element name>.xml . For example: If you use a file called "po.xsd", the name that will appear in the Data source namecolumn will be "purchaseOrder.xml".

To restrict the number of complex elements parsed from a schema (XML views), use the SpecialSchemaSizeLimit setting.

In our next entry, XML Deep Dive with the Magic xpa Application Platform, we’ll more fully explore the detailed usage of XML in Magic xpa. For a detailed look at XML Handling or XSLT for XML Transformations in the Magic xpi Integration Platform see the Integrate My JDE blog which covers JD Edwards ERP integration and a very wide array of related topics.