Web Development

Two ways to import an XML file with .Net Core or .Net Framework

21 November 2019

It's always the simple stuff you forget how you do. For years I've mainly been working with JSON files, so when faced with that task of reading an XML file my brain went "I can do that" followed by "actually how did I used to do that?".

So here's two different methods. They work on .Net Core and theoretically .Net Framework (my project is .Net Core and haven't checked that they do actually work on framework).

My examples are using an XML in the following format:

1<?xml version="1.0" encoding="utf-8"?>
2<jobs>
3    <job>
4        <company>Construction Co</company>
5        <sector>Construction</sector>
6        <salary>£50,000 - £60,000</salary>
7        <active>true</active>
8        <title>Recruitment Consultant - Construction Management</title>
9    </job>
10    <job>
11        <company>Medical Co</company>
12        <sector>Healthcare</sector>
13        <salary>£60,000 - £70,000</salary>
14        <active>false</active>
15        <title>Junior Doctor</title>
16    </job>
17</jobs>

Method 1: Reading an XML file as a dynamic object

The first method is to load the XML file into a dynamic object. This is cheating slightly by first using Json Convert to convert the XML document into a JSON string and then deserializing that into a dynamic object.

1using Newtonsoft.Json;
2using System;
3using System.Collections.Generic;
4using System.Dynamic;
5using System.IO;
6using System.Text;
7using System.Xml;
8using System.Xml.Linq;
9
10namespace XMLExportExample
11{
12    class Program
13    {
14        static void Main(string[] args)
15        {
16            string jobsxml = "<?xml version=\"1.0\" encoding=\"utf-8\"?><jobs>    <job><company>Construction Co</company><sector>Construction</sector><salary>£50,000 - £60,000</salary><active>true</active><title>Recruitment Consultant - Construction Management</title></job><job><company>Medical Co</company><sector>Healthcare</sector><salary>£60,000 - £70,000</salary><active>false</active><title>Junior Doctor</title></job></jobs>";
17
18            byte[] byteArray = Encoding.UTF8.GetBytes(jobsxml);
19            MemoryStream stream = new MemoryStream(byteArray);
20            XDocument xdoc = XDocument.Load(stream);
21
22            string jsonText = JsonConvert.SerializeXNode(xdoc);
23            dynamic dyn = JsonConvert.DeserializeObject<ExpandoObject>(jsonText);
24
25            foreach (dynamic job in dyn.jobs.job)
26            {
27                string company;
28                if (IsPropertyExist(job, "company"))
29                    company = job.company;
30
31                string sector;
32                if (IsPropertyExist(job, "sector"))
33                    company = job.sector;
34
35                string salary;
36                if (IsPropertyExist(job, "salary"))
37                    company = job.salary;
38
39                string active;
40                if (IsPropertyExist(job, "active"))
41                    company = job.active;
42
43                string title;
44                if (IsPropertyExist(job, "title"))
45                    company = job.title;
46
47                // A property that doesn't exist
48                string foop;
49                if (IsPropertyExist(job, "foop"))
50                    foop = job.foop;
51            }
52
53            Console.ReadLine();
54        }
55
56        public static bool IsPropertyExist(dynamic settings, string name)
57        {
58            if (settings is ExpandoObject)
59                return ((IDictionary<string, object>)settings).ContainsKey(name);
60
61            return settings.GetType().GetProperty(name) != null;
62        }
63    }
64}
65

A foreach loop then goes through each of the jobs, and a helper function IsPropertyExist checks for the existence of a value before trying to read it.

Method 2: Deserializing with XmlSerializer

My second approach is to turn the XML file into classes and then deserialize the XML file into it.

This approch requires more code, but most of it can be auto generated by visual studio for us, and we end up with strongly typed objects.

Creating the XML classes from XML

To create the classes for the XML structure:

1. Create a new class file and remove the class that gets created. i.e. Your just left with this

1using System;
2using System.Collections.Generic;
3using System.Text;
4
5namespace XMLExportExample
6{
7
8}

2. Copy the content of the XML file to your clipboard
3. Select the position in the file you want to the classes to go and then go to Edit > Paste Special > Paste XML as Classes

If your using my XML you will now have a class file that looks like this:

1using System;
2using System.Collections.Generic;
3using System.Text;
4
5namespace XMLExportExample
6{
7
8    // NOTE: Generated code may require at least .NET Framework 4.5 or .NET Core/Standard 2.0.
9    /// <remarks/>
10    [System.SerializableAttribute()]
11    [System.ComponentModel.DesignerCategoryAttribute("code")]
12    [System.Xml.Serialization.XmlTypeAttribute(AnonymousType = true)]
13    [System.Xml.Serialization.XmlRootAttribute(Namespace = "", IsNullable = false)]
14    public partial class jobs
15    {
16
17        private jobsJob[] jobField;
18
19        /// <remarks/>
20        [System.Xml.Serialization.XmlElementAttribute("job")]
21        public jobsJob[] job
22        {
23            get
24            {
25                return this.jobField;
26            }
27            set
28            {
29                this.jobField = value;
30            }
31        }
32    }
33
34    /// <remarks/>
35    [System.SerializableAttribute()]
36    [System.ComponentModel.DesignerCategoryAttribute("code")]
37    [System.Xml.Serialization.XmlTypeAttribute(AnonymousType = true)]
38    public partial class jobsJob
39    {
40
41        private string companyField;
42
43        private string sectorField;
44
45        private string salaryField;
46
47        private bool activeField;
48
49        private string titleField;
50
51        /// <remarks/>
52        public string company
53        {
54            get
55            {
56                return this.companyField;
57            }
58            set
59            {
60                this.companyField = value;
61            }
62        }
63
64        /// <remarks/>
65        public string sector
66        {
67            get
68            {
69                return this.sectorField;
70            }
71            set
72            {
73                this.sectorField = value;
74            }
75        }
76
77        /// <remarks/>
78        public string salary
79        {
80            get
81            {
82                return this.salaryField;
83            }
84            set
85            {
86                this.salaryField = value;
87            }
88        }
89
90        /// <remarks/>
91        public bool active
92        {
93            get
94            {
95                return this.activeField;
96            }
97            set
98            {
99                this.activeField = value;
100            }
101        }
102
103        /// <remarks/>
104        public string title
105        {
106            get
107            {
108                return this.titleField;
109            }
110            set
111            {
112                this.titleField = value;
113            }
114        }
115    }
116
117}

Notice that the active field was even picked up as being a bool.

Doing the Deserialization

To do the deserialization, first create an instance of XmlSerializer for the type of the object we want to deserialize too. In my case this is jobs.

1            var s = new System.Xml.Serialization.XmlSerializer(typeof(jobs));

Then call Deserialize passing in a XML Reader. I'm creating and XML reader on the stream I used in the dynamic example.

1            jobs o = (jobs)s.Deserialize(XmlReader.Create(stream));

The complete file now looks like this:

1using System;
2using System.IO;
3using System.Text;
4using System.Xml;
5
6namespace XMLExportExample
7{
8    class Program
9    {
10        static void Main(string[] args)
11        {
12            string jobsxml = "<?xml version=\"1.0\" encoding=\"utf-8\"?><jobs>    <job><company>Construction Co</company><sector>Construction</sector><salary>£50,000 - £60,000</salary><active>true</active><title>Recruitment Consultant - Construction Management</title></job><job><company>Medical Co</company><sector>Healthcare</sector><salary>£60,000 - £70,000</salary><active>false</active><title>Junior Doctor</title></job></jobs>";
13
14            byte[] byteArray = Encoding.UTF8.GetBytes(jobsxml);
15            MemoryStream stream = new MemoryStream(byteArray);
16
17            var s = new System.Xml.Serialization.XmlSerializer(typeof(jobs));
18            jobs o = (jobs)s.Deserialize(XmlReader.Create(stream));
19
20            Console.ReadLine();
21        }
22    }
23}

And thats it. Any missing nodes in your XML will just be blank rather than causing an error.

How to create a shared access signature for a specific folder in Azure Storage

13 November 2019

As we move into a serverless wold where devlopment work is done less on generic multipurpose servers that require patching both to the OS and to the application installed on them, things become easier and in some cases cheaper.

For instance I am currently working on a project that needs to send and recieve XML files from different sources and with no other requirements for a server, Azure storage seemed the perfect fit. As it's just a storage service you pay for what you need and nothing more. All the patching and maintencance of API's is provided by Microsoft and we have an instant place to store out files. Ideal!

However new technology always comes with new challenges. My first issue came with the plan on how we would be getting files in and out with partners. This isn't the first time I've worked with Azure File Storage, but it is the first time I did it without an opps person setting things up. Previously we used FTP, which while dated works with a lot of applications. Like a headphone jack it's not the most glamorus of connection, but it works, everyone knows how it works and everyone has things that work with it. However it transpires that despite Azure storage being 10 years old and reciving requests for FTP from the start, Microsoft have decided to go the same route as Apple with the headphone jack and not have it. Instead the only option for an integration is REST. As it transpires the opps people I had worked with in the past when faced with this issue had just put a VM infront of it, which kind of defeats the point of using Azure storage in the first place!

So we're going with REST and Microsoft provide quite a straightforward REST API all good so far, but how do we limit access? Well there's a guide to Using the Azure Storage REST API which contains a section on creating an authorisation header. It's long and overly complex, but does point you in the direction that to do this you need a Shared Access Signature. The other option is an access key, but this is something you should never give away to a third party.

Shared Access Signature

After a bit more digging through the documentation (and just clicking the thing that sounded right in the potal) I found this documentation on creating an Account SAS which sounded like what I wanted (it wasn't, but it's close).

With a shared access signature you can say what kind of service should be allowed, what permissions they should have, IP address's, start and end dates. All awesome things.

Once I had this I could then use the REST API, but there was a problem. I could access every folder in the storage account and there was no way to stop this! For integrating with 2 third partys they would both be able to access each others stuff, and our own private stuff.

There is also no way to revoke the SAS once it's been generated other than refreshing the access keys which would affect everyone.

Folder Level Shared Access Signature

After a bit more research I found what I was looking for. How to create a shared access signature at a folder or item level and how to link it to a policy.

The first thing you need is Azure Storage Explorer. Once your set up with this you will be able to view all your storage accounts.

From here you are able to browse to the folder you want to share right click it and choose Manage Access Policies.

This will open a dialoge to manage the policies for this specific object rather than the account.

Here you can set all the same permissions as you could for a signature at an account level but now for a specific object and against a policy rather than an actual signature, meaning the policy can be updated in the future with no change to the signature.

Better still you can remove the policy which will then invalidate any signature using it.

For the actual signature key right click the same folder and click Get Shared Access Signature.

Then in the dialoge select the policy from the drop down rather than spcifying the individual permissions.

Click create and you can copy the keys.

You now have an access key that is limited to a specific folder rather than the entire account.

This is only possible to do though one of the code/scripting interfaces. e.g. PowerShell or the storage explorer. The azure portal will only let you get signatures at an account level.

Setting up local https with IIS in 10 minutes

5 June 2019

For very good reasons websites now nearly always run under https rather than http. As dev's though this gives us a complication of either removing any local redirect to https rules and "hoping" things work ok when we get to a server, or setting local IIS up to have an https binding.

Having https setup locally is obviously a lot more favourable and what has traditionally been done is to create a self signed certificate however while this works as far as IIS is concerned, it still leaves an annoying browser warning as the browser will recognise it as un-secure. This can then create additional problems in client side code when certain things will hit the error when calling an api.

mkcert

The solution is to have a certificate added to your trusted root certificates rather than a self signed one. Fortunately there is a tool called mkcert that makes the process a lot simpler to do.

https://github.com/FiloSottile/mkcert#windows

Create a local cert step by step

1. If you haven't already. Install chocolatey ( https://chocolatey.org/install ). Chocolatey is a package manager for windows which makes it super simple to install applications. The name is inspired from NuGet. i.e. Chocolatey Nuget

2. Install mkcert, to do this from a admin command window run

1choco install mkcert

3. Create a local certificate authority (ca)

1mkcert -install

4. Create a certificate

1mkcert -pkcs12 example.com

Remember to change example.com to the domain you would like to create a certificate for.

5. Rename the .p12 file that was created to .pfx (this is what IIS requires). The certificate will now be created in the folder you have the command window open at.

You can now import the certificate into IIS as normal. When asked for a password this have been set to changeit