Getting started with the News API Part 3: Advanced Search

Welcome to Part 3 of our Getting Started with the News API series. In Part 1 we introduced you to the API, the demo/query builder and the supporting interactive documentation. We also showed you how to perform a variety of basic searches to help you get familiar with the API. In Part 2 we began to explore some of the more advanced capabilities that the API has to offer from an analysis and insights point of view, introducing you to the various endpoints available to you. Whether you’re pushing data from the News API into an app, resurfacing the analysis in a news feed or building intuitive dashboards with the extracted data, the News API enables you to obtain a deep understanding for what’s happening in the news, in near real-time. Today we’ll be focusing on some of the more advanced search techniques available to you in the News API, in particular;

Searching by date
Searching by location
Searching and sorting by Alexa Rank
Working with categories

Before you go any further – have you created an AYLIEN account? If not, we recommend you sign up for your Free trial and head back to Part 1 to learn the basics of search, making calls and creating specific search queries. Already created your account and checked out Part 1 and Part 2? Great! Let’s kick on with Part 3, starting with searching by dates.

Making calls:

We’ve created SDKs for some of the most popular programming languages which make using the API super easy. We’ve included example search queries for you to try for yourself throughout the post.

Results:

News API results are returned in JSON format, making it easy for you to do as you please with the data. Throughout this post, we will be displaying charts and graphs that we generated using the JSON results returned from the search query examples provided.

1. Searching by date

The ability to search and source stories according to specific time and date ranges enables you to target news content generated around certain events. As we know, media coverage and public opinion can change day by day, making this search feature all the more important. By pinpointing specific date ranges, we can best analyze the content produced, the opinions held and the sentiment towards our entities and categories of interest.

Example Queries

As an example, and in keeping with our focus on the US Presidential Election, let’s search using the following two queries that are each 3 days in length, but separated by about 3 months; Mentions of Donald Trump between June 16 - June 18 2016 Query

Mentions of Donald Trump in the past 3 days Query

With so much change in the race to become the next president over the past 3 months, the media reaction and variance in results between then and now should provide a very interesting comparison and analysis! Below we compare and visualize sentiment levels from these two time periods. sentitrump Note:The date format required here is a restricted form of the canonical representation of dateTime in the XML Schema specification (ISO 8601). You can learn more about working with dates and XML Schema specification (ISO 8601) in our documentation.

2. Searching by location

The geographic origin of a news story can have a considerable effect on its content, style and writer bias towards certain aspects and opinions. The News API gives you ample flexibility when it comes to working with locations. Whether you want to source stories from a particular location, about a particular location or simply mentioning a particular location, we have you covered. There are three ways to work with locations in the News API; Source Location refers to where the source HQ is located. For example, The Irish Times is based in Dublin, Ireland. Searching Dublin and/or Ireland will therefore return stories retrieved from this source. Source Scope refers to the geographical regions that a source covers. For example, a source may be located in the UK but focuses its content on French politics.The source scope would therefore be France. Mentioned Locations is used to identify when a location as a keyword or entity is mentioned in the title or body of a story. For example, you may want to source stories mentioning Ohio in the title during the US election.

Example Queries

Source Location Stories about US Presidential Elections published by sources from Great Britain

Source Scope Stories about US Presidential Elections published by sources who cover Irish news

Mentioned Location Stories about US Presidential Elections with `New York` mentioned in their title

Learn more about working with locations in our documentation.

3. Searching and sorting by Alexa Rank

By including specific Alexa ranking parameters in your content search you can significantly increase the chances that your results/stories are coming from reputable, popular and trustworthy sources. This is particularly relevant to News API users who are pushing content to apps and news feeds for their own users to consume. Let’s take a look at an example where we want to only search for a specific entity mentioned on sources from a defined range of Alexa rankings.

Example Query

Search for stories that mention Hillary Clinton in the past 7 days from sources with an Alexa rank between 1 and 1000. Query

Screen Shot 2016-09-06 at 15.00.22 The image above is taken from our online Demo using the search query provided. Note the Alexa rank of the The New York Times displayed below the image to the left. We’ve added Alexa integration to nearly all News API endpoints. Here’s a couple of visualized examples using two of the endpoints we covered in Part 2, ‘Stories Over Time’ using the /time_series endpoint and a World Cloud generated from /trends. Screen Shot 2016-09-06 at 15.04.55

4. Working with categories

Knowing the high-level semantic category of stories can be extremely useful in your search for, and analysis of, news content. Our classifier is capable of classifying content into two taxonomies, depending on your preference or use case; IPTC Subject Codes and IAB QAG. In many cases, our News API users rely on category-specific searches to source highly relevant news stories. For example, while a financial news aggregator may have little interest in the daily reportings of a Presidential election, they will certainly be interested in how it affects market performance, consumer confidence and even international relations. They can therefore narrow their search to focus on categories such as Personal FInance, producing relevant stories similar to the one shown below. Screen Shot 2016-09-06 at 15.49.55

Example Query

Mentions of election in the past month classified as Personal Finance under IAB QAG Query

Learn more about working with categories in our documentation.

Summary and Recap

We hope that you have enjoyed our Getting Started guides and have gained a solid understanding of how the News API can be used as a powerful tool in your news content sourcing and analysis. As a quick recap, here are the key points we have covered over the 3 part series; Part 1