Jamie Maguire

Software Architect / Consultant / Developer / Microsoft AI MVP

C#, Cognitive Services, Instagram API, Machine Learning, Social Media

Instagram Graph API – Part 3: How to write a C# API that extracts data using the Insights API

In Part 2 of this miniseries I showed how you can use the Instagram Graph APIs Insights endpoint to extract valuable insights for media shared on the platform for a given account.

We also tested the main methods that ship with the API and extract data such as impressions, reach, engagement and the number of comments.  We built requests using the Instagram Graph API explorer then went onto using Postman to confirm that we could extract data out-with the Facebook and Instagram domain.

To recap, we’re covering the following topics in this 4-part mini-series.

Overview                                       

In this instalment of the miniseries, we’ll write an API in the form of a class library and custom entities that encapsulates functionality which lets you:

  • Handle Instagram Graph API security tokens
  • Builds the required GET or POST requests
  • Execute http requests
  • Deserializes JSON responses from the Instagram Graph API
  • Model data from the Instagram Graph API to DTOs
  • Converts data from DTO’s to custom Entities

By the end of this instalment, you’ll have a class library which:

  • let’s you easily make requests to the Instagram Graph API
  • can shield developers from low-level work that’s often needed when consuming 3rd party APIs
  • can be easily referenced in a software project of your choice
  • can be easily extended to include additional data
  • is easily tested using test automation tools

~~~

Key Requirements and Data

We need to determine the data that we’re interested.  With that in mind, here is the data that our API will let us extract:

  • Media Objects for a given Instagram Business Account
  • Comments associated with each image
  • URL for each image
  • Total number of Likes for each image
  • Total number of Impressions for each image

Note – In Part 2 I identified the raw Instagram Graph API endpoint and showed how to extract this data in Postman.

Project Structure

The API will be housed in a Solution called InstagramPlayground that contains two projects which you can see in the following screenshot:

The detail oriented may have noticed the solution says 3/3 projects, that’s because I had a .NET Core web application that I’ve been using to test the render of Instagram data for an analytics dashboard!

Let’s look at the projects in more detail:

Insta.Graph.API        

This Class Library Project contains two folders, DTO and Logic.

  • The folder /DTO contains classes that represent the messages the Instagram Graph API will return.
  • The folder /Logic contains classes that are responsible for implement our business rules such as creating requests or mapping data from DTOs to our custom Entities.

DTOs and Logic classes that will help deliver the functionality we want our custom API to deliver.

Insta.Graph.Entity

This Class Library Project contains classes that represent the real-world objects that we’re interested in processing.  For example, the Media entity represents an image in Instagram.

Entities sit within the context of the custom API that we’re building.  Any business logic we need to process in our API is performed against an Entity.

Separating DTOs and Entities in this way means we can decouple the API we’re building (and entities it uses) from the messages and datatypes the Instagram Graph API will send.

~~~

DTO’s, Logic and Entities

Here we take a closer look at the components that form our API.  There are 3 main types of class that form the custom API we’re building:

  • Domain Transfer Objects (DTOs)
  • Logic
  • Entities

Let’s look at these now!

Domain Transfer Objects (DTO’s)

We need to represent the messages the Graph API will return.  As data is received it’ll be converted to DTO’s.  These DTOs don’t have any methods or associated business rules.  They simply contain properties that match the data in a response from the Graph API.

You could manually write these DTO’s but the you’d need to create quite a few classes.  Fortunately, there is a quicker way.  You can take the JSON response from the Graph API Explorer or Postman, then copy and paste it into a tool like www.jsonutils.com which will generate C# classes that represent Graph API JSON messages.

Let’s look at the DTO’s in more detail.

Instagram Result, Media and MediaData

These DTOs contain the properties that represent the data our custom API is interested in.  For example, here you can see the properties found in the DTO MediaData:

public class MediaData
{
    [JsonProperty("caption")]
    public string caption { get; set; }

    [JsonProperty("media_url")]
    public string media_url { get; set; }

    [JsonProperty("media_type")]
    public string media_type { get; set; }

    [JsonProperty("comments_count")]
    public int comments_count { get; set; }

    [JsonProperty("like_count")]
    public int like_count { get; set; }

    [JsonProperty("permalink")]
    public string permalink { get; set; }

    [JsonProperty("comments")]
    public Comments comments { get; set; }

    [JsonProperty("id")]
    public string id { get; set; }

    [JsonProperty("timestamp")]
    public DateTime timestamp { get; set; }

    public int MediaDataId { get; set; }
    public DateTime DateCreated { get; set; }
}

Insight and InstagramInsight

This DTO represents raw Insight data that can be extracted from the Insights endpoint. For example, you can extract Media related Insights such as:

  • number of Impressions
  • number of Engagements
  • number of Views

Here you can see the source for the Insight DTO:

public class Insight
{
    [JsonProperty("name")]
    public string name { get; set; }

    [JsonProperty("period")]
    public string period { get; set; }

    [JsonProperty("values")]
    public IList<Value> values { get; set; }

    [JsonProperty("title")]
    public string title { get; set; }

    [JsonProperty("description")]
    public string description { get; set; }

    [JsonProperty("id")]
    public string id { get; set; }
}

… a sample response from the Instagram Graph API that would be mapped to this DTO will be:

{
"data": [
    {
        "name": "impressions",
        "period": "lifetime",
        "values": [
            {
                "value": 349646
            }
        ],
        "title": "Impressions",
        "description": "Total number of times the media object has been seen",
        "id": "18011765518112509/insights/impressions/lifetime"
    }
]
}

I covered most of the data you can extract in Part 2 of this mini-series. You can also find out more information on the Instagram Developer Guide.

Logic

Instagram Manager

We need a class to orchestrate the calls to the Instagram Graph API and manage any business rules we have. The logic class InstagramManager handles this and is responsible for encapsulating all requirements related to:

  • handling security tokens
  • constructing Instagram Graph API requests
  • deserializing responses from the Instagram Graph API
  • parsing JSON data from the Instagram Graph API to custom objects (DTOs)
  • mapping data from DTOs to custom Entities
  • encapsulating the Instagram Graph API calls into easy to use methods

Why Encapsulation?

A quick note on encapsulation.  I’m a big fan of encapsulation and creating application facades for other developers to use. Encapsulating all key business rules in an API is beneficial for several reasons:

  • Maintenance – only place to change core business logic
  • Code Reuse – reference the API in other projects thereby reducing development effort
  • Control – lets you decide which variables are modifiable and which methods can be called externally
  • Automated Testing – encapsulated business rules lend themselves to being simpler to run automated unit tests

For example, years ago, I wrote an API which encapsulated the business rules that were responsible for calculating inmate release dates and parole eligibility dates for the Department of Corrections.  The sentence calculation API was shipped as a DLL that was referenced by Windows Services, Web Services, Data Migration and even Reporting Middleware.

Automated unit tests were written against key methods in the API to ensure that as new requirements surfaced from the business, I had confidence that new code changes didn’t break existing functionality.  Enough about the merits of encapsulating the business rules in an API.  Onto Entities!

~~~

Entities

When data has been returned from the Graph API, we map the DTO’s to Entities.  Entities contain the properties that are needed to satisfy our requirements.  Entities are what our logic layer (Instagram Manager) works with.

What’s wrong with just using the DTOs?

In theory, you could just use the DTOs in the logic layer of our custom API (InstagramManager).  Understandably, you might be wondering why it looks like we’re duplicating code by creating extra classes in the form of entities!

Mapping from DTO’s to custom Entities is a good pattern to implement when connecting to 3rd party REST APIs.  The DTOs sole purpose is to help facilitate the transfer of data between disparate systems, whereas the Entity “belongs” entirely to your domain, or in our case, our custom API.

Some of the benefits this pattern brings can include:

  • promotes the separation of concerns
  • improves code maintainability
  • adaptability
  • future proofing your software

It’s a pattern I implemented in a service-oriented application which had WCF at the heart of it years ago.  One of the projects featured in the solution was class library that contained Data Contracts (effectively DTOs).

These were used by our application and shared with third party agencies that wanted to consume the WCF services I had built.

As users consumed these WCF services, incoming Data Contracts were mapped to Entities within our domain which the logic layer/API then used to apply business rules and process data.

Providing the WCF interface definitions didn’t change, I could make changes to the underlying logic of the application or even the entities without affecting existing clients.

You can read more about some of the benefits of DTOs and Entities here.

Let’s look at the core Entities our API will use in a little more detail.

Media

Here you can see the source code for the Media Entity, the properties are self-explanatory:

/// <summary>
/// Representa a Media object from the Graph API.  Messages/DTOs from the Graph API get converted to Entities as we don't
/// need every property from the deserialized Graph API messages.
/// </summary>
public class Media
{
    public string id { get; set; }
    public int MediaDataId { get; set; }
    public string media_url { get; set; }
    public int comments_count { get; set; }
    public int like_count { get; set; }
    public int impression_count { get; set; }
    public string permalink { get; set; }
    public List<Comment> Comments { get; set; }
    public DateTime timestamp { get; set; }
    public DateTime DateCreated { get; set; }

    public Media()
    {
        this.Comments = new List<Comment>();
    }
}

Comment

…and the source code for a Comment:

public class Comment
{
    public string id { get; set; }
    public string text { get; set; }
}

The entire source code for this tutorial will be on my GitHub profile so you can inspect all of this at your leisure where you’ll find a hierarchical object graph that represents all classes that have been discussed and that are needed by the API we’re building.

~~~

Main API C# Logic (InstagramManager)

Now that we’ve ran through the requirements, process, structure of the projects and main classes that will form our API it’s time to finally look at the code which will let you connect to and run searches against the Instagram Graph API that return image related insights!

To recap, we’re looking to extract the following data:

  • Media (Images) for a given Instagram Business Account
  • Comments associated with each image
  • URL for each image
  • Total number of Likes for each image
  • Total number of Impressions for each image

Process and Workflow

To extract this data, we follow a standard process which consists of the following steps:

  1. Obtaining the security token using the Instagram Graph API Explorer
  2. Identify the REST URL using the Graph API Explorer or Postman
  3. Construct the required request in C# (supplying the security token)
  4. Send the request to the Graph API Endpoint
  5. Process the JSON response from the Graph API
  6. Convert the JSON response from the Graph API to a custom Domain Transfer Object (DTO)
  7. Convert the DTO to an Entity

A picture can express a thousand words, the following UML Sequence Diagram shows you the flow of information and object activations when the InstagramManager Search is executed:

The client actor could me a console application, web application, Azure function or anything that can reference a DLL. As a side point, you’d probably want to flatten out the Entity to a View Model if you were binding the data to a View within a web application.  What follows are the key methods that formed the functionality in the Instagram Manager class:

Constructor

The constructor is straightforward in that it accepts the security token that you have generated from the Facebook / Instagram Developer Console.  This is then assigned to a private variable within the Instagram Manager class:

public InstagramManager(string token)
    {
        if (string.IsNullOrEmpty(token))
        {
            _token = this.access_token;
        }
        else
        {
            _token = token;
        }
    }

Get

This private method is a helper that constructs the core web requests:

private string Get(string uri)
{
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);

    request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;

    using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
    using (Stream stream = response.GetResponseStream())
    using (StreamReader reader = new StreamReader(stream))
    {
        return reader.ReadToEnd();
    }
}

DoMediaSearch

This private method constructs the request that is responsible for fetching the Media objects associated with the Instagram Business Account and other key information:

private List<DTO.InstagramResult> DoMediaSearch()
{
    // get the list of media items
    // parse out the response and the fields we want
    // convert to DTOs and return
    string mediaFields = "media%7Bmedia_url%2Cmedia_type%2Ccomments_count%2Clike_count%2Ctimestamp%2Cpermalink%2Ccaption%7D";
    string mediaSearchUrl = this.baseUrl + mediaFields + "&access_token=" + _token;

    List<InstagramResult> list = new List<InstagramResult>();
    //invoke the request
    string jsonResult = this.Get(mediaSearchUrl);
    // convert to json annotated object
    InstagramResult instagramResult = new InstagramResult();
    instagramResult = JsonConvert.DeserializeObject<InstagramResult>(jsonResult);

    if (instagramResult != null && instagramResult.media != null)
    {
        foreach (MediaData mediaData in instagramResult.media.data)
        {
            list.Add(instagramResult);
        }
    }
    return list;
}

GetMedia

This public method is what consumers of our API can use in order to run a search against an Instagram Business Account.  It returns a strongly typed list of Media Entities.

You can also see this method leverages additional private methods whilst mapping from DTO’s to extract:

  • number of Impressions for each Media
  • lists of Comments for the Media
public List<Entity.Media> GetMedia()
    {
        // invoke the private method - DoMediaSearch()
        List<InstagramResult> instagramResults = this.DoMediaSearch();
        List<Entity.Media> mediaModels = new List<Entity.Media>();

        //map from the JSON/DTO returned by DoMediaSearch() to the Domain Entities
        foreach (InstagramResult instagramResult in instagramResults)
        {
            foreach (MediaData mediaData in instagramResult.media.data)
            {
                mediaModels.Add(
                    new Entity.Media
                    {
                        id = mediaData.id,
                        like_count = mediaData.like_count,
                        comments_count = mediaData.comments_count,
                        impression_count = GetMediaImpressionValue(GetMediaImpressionsInsight(mediaData)),
                        media_url = mediaData.media_url,
                        permalink = mediaData.permalink,
                        Comments = GetMediaCommentsEntities(mediaData),
                        timestamp = mediaData.timestamp,
                        DateCreated = mediaData.DateCreated
                    });
            }
        }
        return mediaModels;
    }

I haven’t detailed the code for other private variables or helper methods such as extracting the Impressions Insight (an int) but you can check that out in my GitHub profile.

Using the API!

We now have everything in place to use the API! To test it I’ve created a .NET Core Web Application which invokes the public method GetMedia:

Invoking the API

You can see how easy it is to invoke the C# API in the following code sample:

InstagramManager instagramManager = 
new InstagramManager("YOUR SECURITY KEY HERE");

List<Entity.Media> instaMedia = instagramManager.GetMedia();

Debugging the API

Eyeballing the debugger shows that our API has been able to successfully grab the data from the Instagram Graph API and populate our respective Entities:

You can see there the number of Impressions (41) and Likes (3) have been retrieved as well as the image URL.  You can also see there is a collection property Comments which contains the associated comments for this image in Instagram (of which there are 4).

Verifying the API data

By taking a copy of the URL from the debug window in Visual Studio and pasting it into the browser we can verify the data is valid, i.e. 3 Likes and 4 Comments:

We’ve successful implemented the functionality!

API Integration with existing software

You might want to integrate an API like this into your existing software application and are wondering where it should live. You’ve got a few options and it will ultimately depend on your business needs.

You could integrate this new API in:

  • Azure Functions
  • Azure Web Jobs
  • Microservices
  • Web APIs (thereby exposing it as service)
  • Web or Windows Applications
  • Windows Services

For example, back in 2017 I wrote a blog that showed how you could simplify your architecture by layering your application.  You can see a screenshot of the overview of this architecture from a web application perspective:

This API would live in the set of Logic components in the above diagram.

As web technology no doubt will evolve (yust think back to the ASMX, WCF days! Now we have RESTful APIs!), you can protect yourself from further rework as anything that sits “above” Logic components can be easily swapped out for other technology, yet the underlying API we’ve built can remain untouched.

Summary and Closing Thoughts

In this article we’ve seen how you can connect to the Instagram Graph API using C#.

We’ve also seen how to:

  • Create a custom API C# with its own set of custom objects
  • create an architecture that is easily extendable,
  • how you can easily map data from 3rd party web services such as the Instagram Graph API
  • invoke, debug and test the C# API
  • looked at how you can integrate the C# API with existing software

The API isn’t complete though and/or you might want to extend it for example:

  • A good practice would be to log each request / response in a database audit table, along with a table to log any exceptions.
  • Remove the magic strings that determine the fields to extract.  Stored these as config options in a database
  • Use Auto Mapper to automatically map data from DTOs to Entities
  • Expose the class library as a REST API
  • Use JWT to secure access and offer as a SaaS service
  • Enrich Instagram data with other data-sources such as Twitter, Reddit, LinkedIn and Facebook
  • Using the C# API to create an analytics dashboard

 

You probably have your own ideas but that’s it for Part 3 of this mini-series!

The next and final part of this mini-series will show you how to integrate Microsoft Cognitive Services and use AI to surface actionable insights in the Instagram data we’ve extracted with the C# API!

You can find full source code for this blog post in my GitHub repo here.

~~~

Are you building software that integrates with social media APIs?

Do you have any further questions?

Is there another topic you’d like to see covered?

Would you like to collaborate on a project?

Do you have any feedback or need support?

Drop me a message below!

JOIN MY EXCLUSIVE EMAIL LIST
Get the latest content and code from the blog posts!
I respect your privacy. No spam. Ever.

2 Comments

  1. Hazem

    Hi like your code, i’m building a software could really you use that code in. do you like to get in touch and see if we can work something out?

    • jamie_maguire

      Hi Hazem,

      Glad to hear that you like the code!

      I will be publishing the final part to this soon (Part 4) which shows you how to leverage AI to surface additional insights.

      Feel free to contact me on email.

Leave a Reply