Pinpoint the news that matters with Entity Prominence and Frequency

Earlier this year we announced ‘Enhanced Entities’, a major step forward for AYLIEN News API. Now, we are further augmenting the capabilities of entities, to help our customers pinpoint the news that matters, by introducing new advanced search features: Entity Prominence and Entity Frequency. Searching for articles using either Entity Prominence or Frequency enables users to retrieve articles that are significantly more relevant to their entities of interest.

Here's a short video using our demo environment showing the value of Entity Prominence:

Now we'll take a deeper look at these new features:

Entity Prominence

What is it?: First, let’s define Entity Prominence. Put simply, it’s a measure of how prominent an entity is in an article, based on:

• If an entity is mentioned in the article title
• How close an entity is mentioned to the top of the article
• How many times the entity is mentioned in the article

Here’s why it matters: Sometimes when we search for news content about specific entities, we end up pulling back stories that are not particularly relevant to the entities that we want to read about. Using Entity Prominence in a query can filter out such stories, effectively separating the wheat from the chaff, leaving only the most relevant stories to our entities of interest. 

For example, if we’re following what’s happening in the news with Microsoft, we may query the News API with a search like ‘entities:{{surface_forms.text: “Microsoft”}}’. Take a look at the top 5 results for this query:

We can see no mention of Microsoft in any of the article titles, meaning a low prominence in the top returned articles. Mentions of Microsoft are most likely incidental in these articles.

However, if we apply the Entity Prominence parameter to the same query, 'entities:{{surface_forms.text: “Microsoft” AND overall_prominence: [0.9 TO *]}}', we see that Microsoft is present in the 5 top returned article titles.

Applying Entity Prominence has clearly had a positive impact on the relevance of our returned articles.

Entity Frequency

What is it?: This feature is simply the number of times an entity is mentioned in an article. Meaning that the more the entity is mentioned, the more important the entity is in the article. We can now leverage this feature to search for articles focussed on our relevant entities. 

If we want to track the recent events around Donald Trump, we previously would have used the following query: ‘entities:{{surface_forms.text: Trump}}’

The titles below are from the top 5 search results given our query. From these titles, there’s not a lot of evidence to support these articles being about Trump himself. This is reflected in the number of times ‘Trump’ is mentioned in each article.

If we leverage the “overall_frequency” parameter in our search, using the query 'entities:{{surface_forms.text: Trump AND overall_frequency: [5 TO *]}}', the returned articles are clearly more relevant to Trump! This is clear by the top 5 returned article titles in the table below.

Like Entity Prominence, Entity Frequency brings the most relevant articles to the fore, meaning we don’t have to sift through articles to find the gold among the gravel. 

AYLIEN News API has the most comprehensive entity capabilities on the market, releasing multiple new enhancements to help users track the entities they care about, with more to come this year. If you would like to learn more about this set of features, or would like to suggest other features, please reach out to your customer success representative. In the meantime, read more about how to get the most out of entities on our doc site, or try a 14 day free trial by signing up here.