XML namespaces

http://www.w3schools.com/xml/xml_namespaces.asp

based on mailing list topics here and here

Dealing with xml namespaces can be a bit difficult in Bots. They can greatly complicate grammars and mappings. Often your EDI partner will want to send or receive XML with namespaces, but actually Bots does not need (or want) them as there are rarely any naming conflicts within a single XML file!

Where an XML file has only one namespace, usually the "default" namespace is used.
eg. xmlns="schemas-hiwg-org-au:EnvelopeV1.0

If the file has multiple namespaces, then namespace prefixes are used. These prefixes can be any value.
eg. xmlns:ENV="schemas-hiwg-org-au:EnvelopeV1.0"

There are several ways to deal with namespaces in Bots:

1. Ignore incoming XML namespace

For incoming XML files that are to be mapped into something else, you probably don't need the namespaces at all. In my opinion this is the "best" way. See Ignore incoming XML namespace for an example preprocessing script to achieve this.

2. Use incoming XML namespace

When a namespace must be used, it is needed in many places (structure, recorddefs, mapping) but you want to specify the namespace string, which can be quite long, only in one place (DRY principle). This has several benefits:

If it needs to change, this is easy to do in one place only
grammar and mapping will be smaller, and look neater

For example, the incoming XML file may look like ...

<?xml version="1.0"?>
<Orders xmlns:xsd="http://www.w3.org/2001/XMLSchema"
     xmlns="http://www.company.com/EDIOrders"
     targetNamespace="http//www.company.com/EDIOrders">
      <Order>
        <OrderNumber>239062415</OrderNumber>
        <DateOrdered>2014-02-24</DateOrdered>
        <LineCount>10</LineCount>
etc...

If you created your grammar with xml2botsgrammar, it probably has the namespace repeated over and over in structure and recorddefs, like this...

structure=    [
{ID:'{http://www.company.com/EDIOrders}Orders',MIN:1,MAX:1,LEVEL:[
    {ID:'{http://www.company.com/EDIOrders}Order',MIN:1,MAX:99999,
        QUERIES:{
             'botskey':      {'BOTSID':'{http://www.company.com/EDIOrders}Order','{http://www.company.com/EDIOrders}OrderNumber':None},
            },
        LEVEL:[
        {ID:'{http://www.company.com/EDIOrders}OrderItems',MIN:0,MAX:1,LEVEL:[
            {ID:'{http://www.company.com/EDIOrders}OrderLine',MIN:1,MAX:99999},
            ]},
        ]},
    ]}
]

recorddefs = {
   '{http://www.company.com/EDIOrders}Orders':[
            ['BOTSID','M',256,'A'],
            ['{http://www.company.com/EDIOrders}Orders__targetNamespace','C',256,'AN'],
          ],
   '{http://www.company.com/EDIOrders}Order':[
            ['BOTSID','M',256,'A'],
            ['{http://www.company.com/EDIOrders}OrderNumber','C', 20,'AN'],
            ['{http://www.company.com/EDIOrders}DateOrdered','C', 10,'AN'],
            ['{http://www.company.com/EDIOrders}LineCount','C', 5,'R'],

# etc...

So, we want to improve this. In your grammar, first add the namespace as a string constant. This is the only place it should be specified, and everywhere else refers to it.

Note: If the XML has multiple namespaces, you can use the same technique and just add more constants (xmlns1, xmlns2, etc).
Also include it in the syntax dict. This allows us to reference it later in mappings.

xmlns='{http://www.company.com/EDIOrders}'

syntax = {
        'xmlns':xmlns,
}

In your grammar, replace all instances of the namespace string with the constant, so it looks like this...

structure=    [
{ID:xmlns+'Orders',MIN:1,MAX:1,LEVEL:[
    {ID:xmlns+'Order',MIN:1,MAX:99999,
        QUERIES:{
             'botskey':      {'BOTSID':xmlns+'Order',xmlns+'OrderNumber':None},
            },
        LEVEL:[
        {ID:xmlns+'OrderItems',MIN:0,MAX:1,LEVEL:[
            {ID:xmlns+'OrderLine',MIN:1,MAX:99999},
            ]},
        ]},
    ]}
]

recorddefs = {
   xmlns+'Orders':[
            ['BOTSID','M',256,'A'],
            [xmlns+'Orders__targetNamespace','C',256,'AN'],
          ],
   xmlns+'Order':[
            ['BOTSID','M',256,'A'],
            [xmlns+'OrderNumber','C', 20,'AN'],
            [xmlns+'DateOrdered','C', 10,'AN'],
            [xmlns+'LineCount','C', 5,'R'],

# etc...

Now in the mapping script, read the xmlns value from grammar.

import bots.grammar as grammar

    xmlns = grammar.grammarread('xml',inn.ta_info['messagetype']).syntax['xmlns']

Then use it wherever needed in the mapping, like this...

    # Get OrderNumber from XML with namespace
    OrderNumber = inn.get({'BOTSID':xmlns+'Order',xmlns+'OrderNumber':None})

3. Outgoing XML with only a default namespace

This can be done by defining xmlns as a tag, and setting it in mapping. It eliminates the need to use namespace prefix on every record and field.
example grammar

recorddefs = {
    'shipment':
        [
        ['BOTSID', 'M', 20, 'AN'],
        ['shipment__xmlns', 'C', 80, 'AN'],   # xmlns is added as a tag (note double underscore)
        ['ediCustomerNumber', 'M', 12, 'N'],
        ['ediParm1', 'M', 1, 'N'],
        ['ediParm2', 'M', 1, 'AN'],
        ['ediParm3', 'M', 1, 'AN'],
        ['ediReference', 'M', 35, 'AN'],
        ['ediFunction1', 'M', 3, 'AN'],
        ['ediCustomerSearchName', 'M', 20, 'AN'],
        ],
}

example mapping

    # xmlns tag for shipment
    out.put({'BOTSID':'shipment','shipment__xmlns':'http://www.company.com/logistics/shipment'})

example output

<?xml version="1.0" encoding="utf-8" ?>
<shipment xmlns="http://www.company.com/logistics/shipment">
    <ediCustomerNumber>191</ediCustomerNumber>
    <ediParm1>4</ediParm1>
    <ediParm2>s</ediParm2>
    <ediParm3>d</ediParm3>
    <ediReference>SCN1022164911</ediReference>
    <ediFunction1>9</ediFunction1>
    <ediCustomerSearchName>SCHA</ediCustomerSearchName>
</shipment>

4. Outgoing XML with default namespace prefixes (ns0, ns1, etc)

This is the default behaviour of the python elementtree module used in Bots. The actual prefix used is not important to XML meaning, so in theory your EDI partners should not care what prefixes you use.

example grammar

xmlns_env='{schemas-hiwg-org-au:EnvelopeV1.0}'
xmlns='{schemas-hiwg-org-au:InvoiceV3.0}'

syntax = {
        'xmlns_env':xmlns_env,
        'xmlns':xmlns,
        'merge':False,
        'indented':True,
}

nextmessage = ({'BOTSID':xmlns_env+'Envelope'},{'BOTSID':'Documents'},{'BOTSID':xmlns+'Invoice'})

structure = [
{ID:xmlns_env+'Envelope',MIN:0,MAX:99999,
    QUERIES:{
        'frompartner':{'BOTSID':xmlns_env+'Envelope',xmlns_env+'SenderID':None},
        'topartner':{'BOTSID':xmlns_env+'Envelope',xmlns_env+'RecipientID':None},
        },
    LEVEL:[
    {ID:'Documents',MIN:0,MAX:99999,LEVEL:[
        {ID:xmlns+'Invoice',MIN:0,MAX:99999,
            QUERIES:{
                'botskey':({'BOTSID':xmlns+'Invoice',xmlns+'DocumentNo':None}),
                },
            LEVEL:[
            {ID:xmlns+'Supplier',MIN:0,MAX:99999},
            {ID:xmlns+'Buyer',MIN:0,MAX:99999},
            {ID:xmlns+'Delivery',MIN:0,MAX:99999},
            {ID:xmlns+'Line',MIN:0,MAX:99999},
            {ID:xmlns+'Trailer',MIN:0,MAX:99999},
        ]},
    ]},
]},
]

example output

<?xml version="1.0" encoding="utf-8" ?>
<ns0:Envelope xmlns:ns0="schemas-hiwg-org-au:EnvelopeV1.0" xmlns:ns1="schemas-hiwg-org-au:InvoiceV3.0">
    <ns0:SenderID>sender</ns0:SenderID>
    <ns0:RecipientID>recipient</ns0:RecipientID>
    <ns0:DocumentCount>1</ns0:DocumentCount>
    <Documents>
        <ns1:Invoice>
            <ns1:TradingPartnerID>ID1</ns1:TradingPartnerID>
            <ns1:MessageType>INVOIC</ns1:MessageType>
            <ns1:VersionControlNo>3.0</ns1:VersionControlNo>
            <ns1:DocumentType>TAX INVOICE</ns1:DocumentType>

5. Outgoing XML with specific namespace prefixes

Your EDI partner may request a specific namespace prefix be used; This is technically un-necessary and bad design, but they may insist on it anyway!

example grammar

xmlns_env='{schemas-hiwg-org-au:EnvelopeV1.0}'
xmlns='{schemas-hiwg-org-au:InvoiceV3.0}'

syntax = {
        'xmlns_env':xmlns_env,
        'xmlns':xmlns,
        'namespace_prefixes':[('ENV',xmlns_env.strip('{}')),('INV',xmlns.strip('{}'))], # use ENV, INV instead of ns0, ns1
        'merge':False,
        'indented':True,
}

structure = [
{ID:xmlns_env+'Envelope',MIN:0,MAX:99999,
    QUERIES:{
        'frompartner':{'BOTSID':xmlns_env+'Envelope',xmlns_env+'SenderID':None},
        'topartner':{'BOTSID':xmlns_env+'Envelope',xmlns_env+'RecipientID':None},
        },
    LEVEL:[
    {ID:'Documents',MIN:0,MAX:99999,LEVEL:[
        {ID:xmlns+'Invoice',MIN:0,MAX:99999,
            QUERIES:{
                'botskey':({'BOTSID':xmlns+'Invoice',xmlns+'DocumentNo':None}),
                },
            LEVEL:[
            {ID:xmlns+'Supplier',MIN:0,MAX:99999},
            {ID:xmlns+'Buyer',MIN:0,MAX:99999},
            {ID:xmlns+'Delivery',MIN:0,MAX:99999},
            {ID:xmlns+'Line',MIN:0,MAX:99999},
            {ID:xmlns+'Trailer',MIN:0,MAX:99999},
        ]},
    ]},
]},
]

example output

<?xml version="1.0" encoding="utf-8" ?>
<ENV:Envelope xmlns:ENV="schemas-hiwg-org-au:EnvelopeV1.0" xmlns:INV="schemas-hiwg-org-au:InvoiceV3.0">
    <ENV:SenderID>sender</ENV:SenderID>
    <ENV:RecipientID>recipient</ENV:RecipientID>
    <ENV:DocumentCount>1</ENV:DocumentCount>
    <Documents>
        <INV:Invoice>
            <INV:TradingPartnerID>ID1</INV:TradingPartnerID>
            <INV:MessageType>INVOIC</INV:MessageType>
            <INV:VersionControlNo>3.0</INV:VersionControlNo>
            <INV:DocumentType>TAX INVOICE</INV:DocumentType>