AI-based hiring tools: Pymetrics

19

September

2021

No ratings yet.

By this time next year, most of you will have graduated from the master’s programme and made yourselves available on the job market. In this blog I want to shed some light on a gamified assessment called pymetrics games that you may possibly encounter during the assessment stage of a job application process.

For those of you who are not familiar with assessments or applying for jobs in general, the traditional job application process (as I have experienced it) generally looks something like Figure 1.

Figure 1. Traditional job application process (via Sierrasoln). Note: steps may vary depending on the type of job or sector.

The traditional (online) assessment stage will generally consist of:

  • ability tests measuring your performance when it comes deductive, numerical and logical reasoning;
  • personality test (questionnaire)

This is where Pymetrics comes in, a company that specialises in developing gamified assessments for recruitment purposes. Companies have opted to fully replace their aforementioned assessment stage with Pymetrics’ patented pymetric games. Some of the most notable companies being: Boston Consulting Group (BCG), JP Morgan, Accenture and Unilever.

What do these pymetric games entail?

The pymetric games are Pymetrics’ core product. It is an online gamified assessment in which candidates have to play through a series of 12 minigames that take two to three minutes each. The assessment uses neuroscience and AI in order to assess a broad range of 91 different cognitive traits. An example of one of the minigames is depicted in Figure 2.

Figure 2. Balloon minigame in which candidates can earn money with every balloon pump. Pumping too much will cause the balloon to explode and make you lose all your money for that respective balloon.

How does it work (in a nutshell)?

Pymetrics creates a custom algorithm for a company by having at least 50 top performers of said company play the pymetric games. Subsequently, this model is used as a benchmark when evaluating applicants’ results. Pymetrics markets its algorithm as entirely bias free, having succesfully subjected the algorithm to extensive AI audits in order to prove their claim.

So what’s the catch?

As my fellow peers Andrew Tan and Tamas Vincze have already explained in great detail: algorithms are inherently biased. In addition, an independent AI audit of Pymetrics’ algorithm found that although it passed the formal checks, the audit itself did not prove that the tool is bias free nor that it actually picks the most qualified candidates for a job.

This brings me to my question: how do you feel about an AI-based hiring assessment being put into practice? Would you much rather prefer the traditional online assessment? Having personally experienced both types of assessment, I am curious to see where my fellow peers stand, especially as you prepare yourselves for your job search.

References

Did this blog pique your interest? Please refer to the used sources below for more in-depth coverage on this topic:

  • https://www.technologyreview.com/2021/02/11/1017955/auditors-testing-ai-hiring-algorithms-bias-big-questions-remain/
  • https://digital.hbs.edu/platform-digit/submission/pymetrics-using-neuroscience-ai-to-change-the-age-old-hiring-process/
  • https://www.graduatesfirst.com/pymetrics
  • https://hackingthecaseinterview.thinkific.com/pages/bcg-pymetrics-test
  • https://www.jobtestprep.com/pymetrics-games#balloon-game
  • https://sierrasoln.com/hiring-process/


Please rate this

Extending the Concept of Versioning in the Video Games Industry

7

October

2018

How versioning has impacted the gaming industry and how companies use it to capture value long after a game is released.

5/5 (2)

Building on Tom’s article which can be read in this blog (https://digitalstrategy.rsm.nl//2018/09/21/versioning-and-the-gambling-debate-in-the-video-game-industry/), the video games industry is perhaps one of the best examples in order to understand the dynamics of versioning and to grasp its various benefits. In this sector, versioning can be accomplished with little effort and in varied ways, normally with different strategic purposes and allowing companies to derive various benefits.

The first and most evident way is perhaps, multiplatform releases. In order to cater to a wider audience, publishers release their games in multiple platforms (generally, Xbox One, PS4 and PC), mostly with minimal observable differences between them. Adding to these already three different versions, publishers also usually launch pre-order bonuses and different versions for each platform. Taking the FIFA 19 game example, at launch you could choose from three versions for each platform: Standard Version containing the full-game (€69.99), Champions Edition containing the full-game, 3-day early access to the game and FIFA Ultimate Team (FUT) boosts (€89.99) and finally, an Ultimate Edition with all of the above benefits plus additional FUT boosts for €99.99. This said, upon release only, FIFA 19 had already launched in nine different versions.

Other times, versioning can be integrated into the game’s business model from day one and be the source of revenue, as it happens with the freemium model, similarly to Spotify’s business model, which distinguishes between a free and a premium version. However, long after release, companies are still able to capture value from versioning, being able to do it mainly through three different ways: complete editions/game of the year editions, remakes and remasters.

A videogame’s lifecycle has widely changed during the last generations of consoles and before, as games would become outdated and their player-base would decrease, months or a couple of years after release, the games could be found heavily discounted on stores. This has changed, however, with the introduction of Digital Local Content (DLC) which adds additional content to the original game under the form of features, items, quests or modes attracting players back into the game. After releasing these packs of contents, companies are now able to bundle them together with the initial release under a name similar to “Complete Edition”, bumping the price of a one- or two-year-old game back to the full-price of 69.99€ at a low marginal cost.  A good example of this is Ubisoft’s Assassin’s Creed 2, which has been bundled with DLC and re-bundled with other games not in one, not in two, but approximately in fourteen different versions.

Versioning can also come from the remastering of a game, which consists in relaunching the existing game while making some small tweaks (e.g. increase resolution from 720p to 1080p or from 30 frames-per-second to 60) or introducing new features in order to take advantage of the capabilities of a new platform. This versioning strategy presents several advantages to developers and publishers. Firstly, the costs of remastering a game are significantly lower than developing one from scratch and its production process can usually be outsourced. Secondly, it allows companies to promote their intellectual properties (IP) to audiences that were not able to purchase their IPs before as they, for example, did not have the console for which it was initially launched. Thirdly, these remakes are commonly launched right before a new game of the series is released, proving as an effective way to promote it. Finally, when new hardware is launched, remasters allow developers to experiment and learn from it, representing a much less risky approach then releasing a brand-new big budget title.

Companies can also remake their games, which is not to be confused with remastering since remaking (as the name implies) consists in remaking a game almost from the ground up rather than simply tweaking it. This usually happens if the game becomes too outdated, making it very difficult for developers to update existing assets and to use an obsolete game engine, making it more efficient to just remake it all together. This strategy has the added advantage of appealing to the already existing core fanbase, leveraging the “nostalgia economy”.

This said, ways of versioning in the video game industry are wide and diverse, and should be integrated into a company’s strategy accordingly as a way to capture value during or long after a product is released. What do you think? In what other ways can developers and publishers capture value through versioning? Do you have a better example of an industry where versioning is an expanding reality? Let us know in the comments!

 

Please rate this

Gamers, analyzing your big data for you?

5

October

2017

5/5 (1) Do you ever play video games? Do you see yourself as a gamer? Yes? No?

 

Whatever your answers are to these questions, you must’ve heard of the discussion on the influences of certain video games on its users. A 2015 survey by PEW Research Center showed that 76% of adults do not agree with the statement that “most games are not a waste of time” and only 11% of adults agree that “video games are a better form of entertainment than TV” (Duggan, 2015). Now I do agree that many video games are not adding any value to anyone else but the gamer, but we could take the step to change this around a full 180 degrees. Digital technologies have already enabled multiple initiatives to do so. Let’s take a quick look at two examples.

Have you ever heard of the game Genes in Space? It is a game developed by Cancer Research UK to help them analyze their data retrieved from gene microarrays. Over 400,000 people worldwide played the game and were able to collectively analyze close to 5,000,000 nuggets of data, which represented the entire genomes of 1980 patients, each checked 50 times for accuracy (Cancer Research UK, n.d.).

 

 

A second game is Foldit, an online puzzle game from 2011. This game about protein folding helped researchers to “resolve the structure of an enzyme that causes an Aids-like disease in monkeys” (Mohammadi, 2014). Gamers solved this puzzle in three weeks, after researchers had already spend 13 years on it.

 

 

There are far more games alike these two, enabling anyone to help science. But why are companies not yet investing in creating games that would help them analyze their big data? In a time where the amount of data created is doubled at least every two years, the pressure on big data analysts is high. Adding to that that 40% of companies are struggling to find and retain data analytics talent (Business.com Editorial staff, 2017) would make it seem that any help available can be deemed useful.

So wouldn’t it make a lot of sense if, especially large, companies start to invest their money in finding ways to feed their big data into games? Assuming that such a game could be made attractive to play,  they would potentially be able to reach millions of gamers around the globe who can help them finding patterns in their data and understand the data better. Especially since we know that we spend around 3 billion hours a week playing online games (Mohammadi, 2014), the potential could be enormous. The data could then be replaced by objects or other things in a game to protect the data itself, as almost no company would want to have their big data open to the public. For example, in the game Genes in Space, the gene data was masked in the form of a substance called ‘Element Alpha’.

Could you imagine such a development to come, or do you know of any way this is already being applied? Or do you see major problems that would make this impossible? I would be very curious to hear what your thoughts are on this.

 

References

Business.com Editorial staff. (2017, February 22). Big Data, Big Problem: Coping With Shortage of Talent in Data Analysis. Retrieved from Business.com: https://www.business.com/articles/big-data-big-problem-coping-with-shortage-of-talent-in-data-analysis/

Cancer Research UK. (n.d.). The Projects. Retrieved from Cancer Research UK: http://www.cancerresearchuk.org/support-us/citizen-science/the-projects#citizenscience1

Duggan, M. (2015, December 15). Gaming and Gamers. Retrieved from PEW Research Center: http://www.pewinternet.org/2015/12/15/gaming-and-gamers/

Mohammadi, D. (2014, January 25). How online gamers are solving science’s biggest problems. Retrieved from The Guardian: https://www.theguardian.com/technology/2014/jan/25/online-gamers-solving-sciences-biggest-problems

Please rate this

Five Games For A Dollar – How Humble Bundle Uses Price Discrimination

24

September

2016

No ratings yet. Do you remember the time when you had to pay $50 for a game in the store? And the price did not change much even after a long time. How things have changed since then. Ever since games are available in fully digital form on platforms such as Steam, prices have gone down a lot (even though they often still start out as expensive as in the past). How is it possible that games have become so much cheaper? Does the gaming industry simply make less profit? This is possible, but not necessarily the case. The answer lies in the fact that games are an information good, which means that while it is very expensive to produce the first copy, creating more copies is relatively cheap. So the more copies you manage to sell, the smaller your total cost per copy is (because the production cost is spread out over more products). In other words: to maximize your profit as a game developer, you want to sell as many copies as possible, even if this means reducing your price. Now, keeping this in mind, let us consider the concept of price discrimination: each individual has a willingness to pay for a particular product. Some people might be willing to pay $50 to get a game as soon as it is released. However, other people may only want buy that same game for $1. Ideally, you’d offer the game to the one person for $50 and for $1 to the other. However, if the game is sold in a store, you cannot differentiate the price, which is why in the past the only way to discriminate prices for certain groups of individuals was by changing the price over time, so that those with a lower willingness to pay would wait until the price dropped. But even then, putting a game in a store is obviously more expensive than selling a digital, downloadable version online, such as what Steam offers.

 

This is all very interesting of course, and a very happy development for all gamers, but this is not the phenomenon that I wanted to talk about. I’d like instead to point out another website that sells games: Humble Bundle. They have a very interesting business model where they have extended this idea of willingness to pay and combined it with the idea of bundles. What Humble Bundle does is they create a bundle of games and offer it for about a week. Now what is special is that you can pay what you want! In fact, you could get many of the games they offer for free (although you must pay at least $1 to get Steam keys, allowing you to download the games in Steam). What’s more: you can choose where your money goes. By default, part of it goes to each game developer, part goes to Humble Bundle itself and part goes to charity. But you can customize this any way you want, choosing not to give anything to Humble Bundle, or giving everything to a charity. In this way, you have the possibility to support the developers of the games that you really like. So why does this concept even work? If you are giving away games for free, how can you possibly make a profit? Well, obviously most people choose to pay a little bit, either to get access to the games on Steam or to ease their consciousness. In addition, Humble Bundle smartly gives an incentive to pay more by allowing you to ‘upgrade’ your bundle by paying more, meaning that other (usually more recent or more popular) games are added. This works because, as mentioned earlier, the Humble Bundle system allows great personalization, thus making it possible to make maximum use of each individual’s willingness to pay (and some people do in fact pay over $50 for a bundle). In addition, the idea of making bundles is very smart. People will feel that the total value is more, even if they only end up playing one game in the bundle. After all, it is easier to motivate someone to buy that one game that they want for $15 even though their willingness to pay was $10, since they get that other game that they were kind of interested in as well. Not to mention that a great way to sell unpopular games is by combining them with more popular games. Even if people can choose where their money goes, they likely won’t often choose to give nothing at all to a particular company.

 

All in all, I believe Humble Bundle’s business model is very smart, and while it may at first seem counter-intuitive, it likely leads to quite a bit of profit for game developers. In addition, they have executed this idea, that can be brought back to various economic theories, in a really good way. I hope they will continue to experiment and find new ways to make both consumers and developers happy. Now go celebrate this great development by treating yourself to a bundle of games – for whatever you’re willing to pay!

Please rate this