AI Day 2023: Highlights from the Annual HR Tech AI Showcase

Artificial intelligence (AI) continues to forge its place in both our personal and professional lives. But not all AI is created equal. 

At Phenom, we believe AI needs to draw from rich context to deliver the experiences that candidates and employees expect with the precision that empowers HR teams, employees, and managers to reach superhuman levels of efficiency. 

Our second annual AI Day explored the intricacies of this approach by highlighting the:

• Latest AI advancements in HR 

• Hiring and retention challenges they solve 

• Innovations that lie ahead

Over the course of 4+ hours, our domain experts provided a technical deep-dive that spanned some of the biggest HR topics today, including: the data models powering HR technology; bias and ethics; and AI’s impact on candidates, employees, recruiters, managers, HR, and HRIS teams. 

Read further to get key takeaways and discover how AI has been designed to maximize HR team efficiency while keeping humans in the driver’s seat. Want more than just the highlight reel? Check out the full-length, on-demand replay here. 

The State of AI – and What this Means for HR

We all know that AI has been interwoven into the background of our experiences — from Amazon product recommendations to suggested shows to watch on Netflix. But recently, the hype around AI has been driven by newly adopted Generative AI platforms, like ChatGPT, and the foundational models that make those easy-to-use features possible. 

But underneath all the hype and news coverage, there have been elements of AI that have plateaued and become stable in the last 10 years, said Mahe Bayireddi, our CEO, during his opening remarks. For example, autonomous cars, dynamic content, and knowledge graphs have become stable outputs from AI. 

But what does all of this mean for HR, and how can companies continue to implement AI to achieve their goals faster, smarter, and more efficiently? 

According to Bayireddi, there are four key areas that will help paint the picture of what AI means to HR in the coming years:

1. Skills will change dramatically, altering the work landscape

2. AI is an assistant — not autonomous — especially in HR 

3. The balance between AI and humans is essential, important, and critical

4. AI regulation can be modeled after other high-stakes industries (e.g., nuclear, climate change, aviation)

You might be wondering, “What about right now? How can I use Phenom AI to impact my every day?” By understanding where AI is going, our teams spent the time to create a scalable solution that addresses the problems of today — and tomorrow. 

To do so, we custom-built intelligent frameworks, databases, guardrails, and infrastructures specifically for HR and talent experiences using billions of data points compiled from proprietary Phenom platform data. 

This ensemble approach helps keep innovation at your fingertips, giving you a competitive edge when it comes to attracting, engaging, and retaining talent. Keep reading to explore how our custom HR solutions and meticulously trained AI benefit your teams and evolve alongside your business using human-in-the-loop (HITL) feedback.

Solving Real World HR Problems with Generative AI

Generative AI (or GenAI) is transforming the way we work. Its most well-known application is assisting in the content creation process. Our teams took that benefit one step further by leveraging the tool to offer various streamlined processes throughout different stages of the hiring journey. 

Within the Phenom platform, GenAI can:

• Summarize candidate profiles and notes from interviews to give recruiters more detailed information about each candidate

• Identify succession planning opportunities for high-performing employees

• Automate interview transcripts to allow interviewers to focus on the interview instead of taking notes 

• Detect skills gaps and surfaces upskilling and reskilling opportunities

• Analyze and summarize key points based on interview transcripts, providing meaningful evaluations of candidates 

• Generate personalized emails and campaigns to evolve employees 

• Reference career site pages to provide the platform bot with more context to answer questions from site visitors 

• Create audience segments to personalize and scale development efforts 

• Generate content on the fly to assist with targeted messaging for campaigns based on intended audiences 

And those are just a couple of examples of how we’re integrating GenAI into the platform to support your teams across both talent acquisition (TA) and talent management (TM).

Now let’s take a look under the hood to explore the infrastructure of Phenom AI that makes GenAI use cases possible.

A look inside Phenom’s Generative AI Infrastructure

Phenom’s technology encompasses hardware, software, and security layers. Within these layers, there are numerous frameworks, databases, models, and experiences. The following components comprise the infrastructure that allows users of the Phenom platform to interact with GenAI: 

• Experience layer 

• Data and Trust layer

• Large language learning models (LLMs)

All of these elements work together to provide critical enterprise context that is enriched with each interaction that occurs throughout our Intelligent Talent Experience Platform.

But our teams didn’t stop there. To ensure that our AI has the necessary data points required to allow it to obtain critical context surrounding each recommendation, decision, profile, and user, we trained our own LLM for HR-specific applications. 

This decision stems from our commitment to addressing the unique challenges and intricacies of the HR domain. The development teams achieved this by fine-tuning a Llama-2-7b model using an extensive dataset of 50 billion tokens from the Phenom domain.

Now, let’s take a look at another layer of context that helps decipher these massive sets of data in a way that helps TA and TM teams identify, engage, nurture, and retain top talent within your organization.

The Intelligence Layer: Building the Right Infrastructure To Scale Phenom AI

Typically, location, skills, job titles, and additional information are passed to a platform in multiples because this data is getting pulled into the system from numerous endpoints — think of your ATS, resume submissions, manual imports, etc. 

This data is beneficial, but only if it’s standardized and segmented appropriately. That’s where our data standardization model becomes integral. Data standardization is a critical area that provides our products with a uniform understanding and richer context about the specific data attributes and how they relate to each other. 

Once the attributes are cleaned up and standardized before being integrated into our platform, our universal knowledge graphs and recommendation systems can leverage the clean data and understand the numerous variations. 

But before we created systems that can operate on this level and parse accurate understanding of pertinent data, we had to train our models to do so. 

This process involved building standard datasets and taxonomies that ensure AI can interpret and contextualize the massive data sets effectively and accurately. We combined custom potentially unwanted program (PUP) data with data standardization, normalization, and augmentation — or DNA for short — to curate and maintain standard data sets and provide services to standardize entities. 

To take these in-depth insights to another level, we combine our platform knowledge graphs with enterprise talent graphs so we better understand your enterprise, your roles, your infrastructure, and your users. 

But why do we do this? Our goal is to build knowledge graphs at the enterprise level to offer maximum context about a client that can be used for recommendations and insights throughout the platform, ultimately driving a more personalized experience. 

It also certifies that our platform and infrastructure can scale and evolve alongside your organization, delivering better insights every day.

The Benefits of a Talent Data Platform

Although enterprise talent graphs and context are an integral piece of the puzzle, understanding how HR at your organization operates on a daily basis is much more complex. That’s why we paired enterprise context with a talent data platform (TDP) that provides a 360-degree view of individual talent which enables us to offer advanced AI solutions — like real-time personalization and deeper candidate insights for richer decision-making for all personas. 

The Phenom Intelligent Talent Experience platform captures data about individual talent across products like a Career Site, Talent Candidate Relationship Management (CRM) software, Applicant Tracking System (ATS), Talent Employee Relationship Management (ERM) software, and others, alongside data belonging to various personas such as candidates, recruiters, and employees. However, the identities are distributed. TDP is aimed at addressing the challenges arising from current systems to create a better overall view of candidates within your talent pipeline and your organization.

TDP brings in Intelligent Identity resolution with human feedback loops for conflict resolution. This is followed by configuration-driven consolidations and data governance. Finally, it provides seamless integration into People Analytics and Data Engine to accelerate data usage. 

So what does that mean for you and your teams? It means that instead of viewing candidate profiles from one or two lenses and assessing them in a silo, our platform views that data from all sides and assesses how the information within those profiles relate to one another, creating a universal understanding of the available data that can be applied to recommendations — effectively accelerating actions and decision-making processes.

Going From Talent Scarcity to Prosperity with AI

“At Phenom, we don’t just provide career sites — leveraging AI, we curate personalized experiences, matching the right candidate to the right job,” said Lindsay Mareau, Vice President of Strategy at Phenom. 

To continually deliver unparalleled experiences for all users within our platform, our teams have prioritized building intelligent systems that account for the nuances of HR and talent data — because after all, HR is personal. 

We do this by leveraging Phenom BERT, O*NET classifications, and education taxonomy classifications to help our AI interpret context accurately, appropriately, and efficiently. Here’s a quick overview of each of these systems: 

• Phenom BERT: a bidirectional encoder transformers-based language model trained specifically on the HR domain based on 24 million training samples across all 23 job families, creating 2 billion tokens for training. Our fine-tuned Phenom BERT model is three times smaller than the generic BERT model, accelerating training capabilities to achieve enhanced accuracy and metrics, improved contextual understanding, and elevated efficiency and adoption. 
  

• O*NET Classifications: designed to bridge the gap in job-candidate matching by reorganizing jobs for accurate recommendations while fostering AI transparency and explainability using job titles, skills, and job descriptions. Rolling out this solution drove a 12% increase in conversion rate in AI discovery and a 5% increase in conversions for similar job recommendations. 
  

• Education Taxonomy Classifications: this solution was driven by the need to address the problem of training an Education Taxonomy Model based on the International Standard Classification of Education, which is maintained by UNESCO. Our hierarchy follows their guidelines but we adjusted our training model to focus on the top-level and third-level classifications to provide a comprehensive understanding of educational qualifications within the platform. This allows our AI to draw better conclusions when parsing educational backgrounds and skill sets. 

All of these systems work in parallel to give our product features the context needed to recommend quick actions, identify best-fit candidates faster, and reduce time spent on repetitive tasks, allowing your teams to win the race to engage talent. 

Let’s take a closer look at a new development that has been underway for the last two years and will directly impact how effective your recruiters can be. 

Taking Career Sites Search to a New Level  

Traditional career sites lack advanced AI capabilities, limiting their ability to provide precise, relevant matches and optimized user experiences. 

To solve this problem, we developed Phenom AI-powered Intelligent Search. This feature was designed to improve the 8M searches that are happening across our platform every day. Our Intelligent Search allows for: 

• Typeahead search

• Enriched intent detection 

• Auto synonym detection 

• Personalized results 

• Dynamic re-ranking 

• AI adaptive learning 

• Rules engine 

• Built-in A/B testing

But again, we didn’t stop there. We paired this enhanced search functionality with new elements of personalization to create a well-rounded, seamless, and user-friendly experience for both recruiters and candidates. 

Using Audience Personalized Widgets (AWPs), site visitors can expect a new career site experience almost every time they visit. AWPs are engineered to actively interpret and respond to the specific user, including a stranger, visitor, lead, qualified, or applicant. 

Depending on each user category, these widgets populate relevant information to communicate available roles, recommend content, highlight recently viewed jobs, and more. Consistently delivering new content through an AWP ensures candidates receive topical information that’s relevant to their interests, allowing you to meet them where they are.

What About Conversational AI and Chatbot? 

To support an intuitive and personalized career site experience, chatbots outfitted with conversational AI serve as always-on assistants to your recruiters. Today, most chatbots are driven by workflows, outfitted with prefilled answers to a set number of frequently asked questions, and are only available on the web — leading to chatbot experiences that leave a lot to be desired.

Specifically, our teams found that chatbot conversion funnels are suboptimal, the job views-to-applications ratio is too low, and there are a high number of unanswered questions. To fix these issues and increase the efficacy of your career site chatbot, we revolutionized conversational AI. 

Now, the Phenom Chatbot is equipped to: 

• Have natural, human-like conversations instead of flow-driven conversations

• Provide contextual answers to questions by combining multiple knowledge bases and user-level contexts

• Offer omnichannel experiences, enabling users to continue the conversation anywhere at any time

• Empower a multilingual experience that allows people to converse in their preferred languages 

Our teams have left few stones unturned when it comes to creating stellar experiences for all of our platform users, ensuring your organization can make a great impression on talent throughout every stage of the talent lifecycle. 

Unlocking Recruiter and Hiring Team Efficiency 

The old way of searching for talent is outdated, which is one of the many reasons HR teams are adopting AI to streamline their recruitment process. Today’s modern approach that transforms old methods of searching for talent is Intelligent Sourcing.

Intelligent Sourcing is deeply rooted in the effort to improve the relevance and effectiveness of the search process by leveraging ML algorithms to re-rank the search results. With the goal of improving search relevance, personalizing search results according to recruiter user context, and improving search conversion rate, it was important for our team to build a model that adapts to different data points and inputs over time.

By creating a two-phase solution in the search flow, our development teams improved the entire framework — and as a result, search capabilities and outputs are enhanced. Our new search process is enriched with Intent/Entity Extraction and Recruiter Taste Graph implementations. This enhances the search experience by returning more relevant results based on the context of the search terms and the typical behavior of the recruiter. 

We also implemented Learning to Rank (LTR) model in the post-search phase which focuses on re-ranking search results to efficiently prioritize job listings within a vast database of content. Pairing this integration with our in-house trained Phenom BERT created input feature vectors for the ranking LTR model. This allows the AI to consider candidate features, job features, and recruiter features when generating search results. The increased access to relevant data helps recruiters narrow down which actions need to be taken first, second, and third, allowing them to maximize their time while making more informed decisions. 

We coupled this updated search model with our recruiter-facing X+ Bot to assist with everyday activities, like adding notes to candidate records, and answer any questions recruiters may have when moving throughout the platform, like learning how to create spotlights. Both of these developments make searching, sourcing, and contacting best-fit candidates more efficient and accurate.

Let’s take a look at how our team tackled another problem area for recruiters and designed a solution that’s making a positive impact on key hiring metrics. 

Scoring and Matching for Knowledge Work Using AI Discovery and Fit Score 

Recruiters today face numerous challenges when hiring for their organizations. It can be time-consuming to manually sift through an overwhelmingly large pool of candidates to connect the best-fit talent with available roles. Plus, the longer it takes to identify potential candidates, the greater the risk that the company will lose that talent to a competitor who can connect with them faster. 

Fit Score — a dynamically generated score based on desired skills, title, experience, and location — can solve many of these bottlenecks the AI way. And when paired with AI Discovery, recruiters can achieve superhuman levels of efficiency with confidence. 

AI Discovery quickly shortlists candidates based on their qualifications and how well those data points line up with specified job requirements. This feature starts functioning in the background the moment a job has been created. 

AI Discovery recommendations consist of internal and external candidates that are already in your CRM, and talent outside your CRM that can be added to specific jobs — offering a full breadth of potential options to help recruiters find the best fit for each open position. 

Over time, these recommendations change based on feedback and recruiter preferences, which allow for continuous evolution. Then, candidates are organized into a shortlist, quickly highlighting the top options that recruiters should look at first. This shortlist is then assigned a Fit Score that visually highlights which individuals in that shortlisted group are A, B, or C fits based on job criteria. 

It’s important to note that Fit Scores aren’t arbitrary. They’re generated after analyzing numerous data points, including but not limited to previous skill sets held by individuals hired to that role in the past, geographic location fit, job title relevance, and much more. Once Fit Scores are assigned, the human-in-the-loop process begins, and recruiters can provide feedback on how well a candidate fits the criteria they’re looking for. These actions help the AI make more accurate predictions in the future. 

Although Fit Score has already proven effective, our teams wanted to enhance its capabilities further. Our updated Supervised Fit Score has client-specific neural network models, which score a candidate based on past hiring decisions. Recruiter actions in the CRM, like additions and dismisses of candidates, are also sent to the model for training. We tested three different models within our Supervised Fit Score testing: a Triplet model, a CNN model, and a SupCon model. 

Let’s look at the results: 

Keep reading to learn how we apply our AI to another element of our platform — our Automation Workflow Engine — to drive improvements that translate to time and cost savings for our customers.

Leveraging Automation for Proactive Hiring 

HR teams struggle to build efficient and effective processes at scale. At best, they’re typically only able to construct automations within siloed point solutions. That’s where our Automation Engine comes in. We build personalized and highly automated workflows for all HR processes to streamline activities, alleviate redundancies, and save time so talent teams can focus their energy on the more strategic aspects of hiring. 

For example, Automation Engine can automate tasks like:

• Posting a job requisition

• Scheduling and rescheduling interviews

• Sending candidates follow-up communications

• Sharing job templates with hiring managers,

• Notifying recruiters when responses are input

• Providing recruiters with a candidate list

• Notifying you when qualified candidates apply

When powered by AI technologies, Automation Engine can also rank candidates based on psychometric assessments, auto-schedule interviews for eligible candidates, and handle communications with hiring managers.

Here’s a list of the most frequently used automation recipes across the platform: 

• Send an email

• Update candidate field

• Add to campaign

• Add tag

• Send SMS

• Add to list 

• Change hiring status 

Did we mention that automation recipes can be combined to handle end-to-end use cases? By leveraging our AI-powered Automation Engine, your repetitive systems can run on repeat automatically, saving your team's time so they can focus on making meaningful connections with candidates before your competitors do. 

 With all of these innovative and intuitive AI applications, you might be curious about the implications of AI when it comes to compliance and privacy. Keep reading to find out how we apply cutting-edge compliance to cutting-edge AI.

Navigating AI Compliance

We want to make it easy for you to adopt AI and leverage it responsibly to achieve impactful outcomes for your business. A large component of this relies on our ability to ensure AI compliance. 

Our teams are detailed and meticulous when assessing our product features for compliance with privacy and compliance requirements. Here’s a list of the areas of compliance we abide by within the Phenom platform: 

• Office of Federal Contract Compliance Programs (OFCCP): employment-based internet applicant rule mandating appropriate recordkeeping with EEO reporting guidelines that mandate data collection on applicant demographics. 

• Uniform Guidelines on Employee Selection Procedures (UGESP): employment-based compliance guidelines designed to assist employers, labor organizations, employment agencies, and licensing and certification boards to comply with requirements of Federal law prohibiting employment practices that discriminate on grounds of race, color, religion, sex, and national origin. 

• NYC Local Law 144: AI-based compliance detailing that automated employment decision tools must be audited and the audit results are required to be disclosed. 

• EU AI Act: AI-based compliance that aims to ensure AI systems placed and used on the Union market are safe and respect existing laws on fundamental rights and Union values. 

• California Consumer Privacy Act (CCPA): privacy act that requires companies to comply with user requests for all data collected and stored, each category of sources where data is collected, the business purpose of collecting and selling user data, as well as a list of third parties that have access to a user’s data. 

• General Data Protection Regulation (GDPR): EU privacy law that dictates an organization that falls within the scope of the GDPR meets the requirements for properly handling personal data as defined in the law.

Additionally, our teams conduct risk assessments following our Governance Policy for AI technologies. This policy is based on the Model Artificial Intelligence Governance Framework originally developed by the government of Singapore for the World Economic Forum and is similar to other AI governance approaches, such as Google AI Principles. 

Under the Phenom Governance Policy, AI-driven technologies are assessed based on their probability and severity of harm, prospective mitigations, and need for humans in the loop. 

Based on this policy and the desire to accommodate your preferences and needs within the platform, you can:

• Enable or disable Fit Score to fit within location jurisdiction requirements 

• Offer an opt-out functionality to applicants so they can decide whether they’re comfortable having an AI-based tool evaluate them

With the legal definitions out of the way, keep reading to explore how we take those policies yet another step further to ensure AI safety at scale using Industrial/organizational psychology and other principles.

AI Safety at Scale 

Our teams focus on the importance of ethical hiring at the enterprise scale. “All good AI needs an off button, a control panel, a monitoring tool, and success metrics to ensure you’re meeting your goals,” said Phenom’s James Schlitt, Ph.D., Senior Data Scientist during AI Day. 

To do this, we focus on the following areas of concern:

• Valid AI: the degree to which artificial intelligence’s predictions or decisions are accurate, reliable, and appropriate for a particular context or purpose. 

• Fair AI: the application of algorithms and machine learning models in a way that they provide equal opportunities, avoid discrimination, and prevent biases based on certain attributes such as race, gender, or age. 

The last area of concern is bias, but there are multiple ways to define bias based on statistics, machine learning, and Industrial/Organizational (I/O) Psychology.

• Statistical Bias refers to systematic errors that skew findings or conclusions. It happens when the process of collecting data or the methods of analysis favor certain outcomes over others, leading to inaccurate representations of populations or phenomena.

• Machine Learning Bias is a phenomenon that occurs when an algorithm produces results that are systematically prejudiced due to erroneous assumptions in the ML process. 

• I/O Psychology Bias is any complex of unfair or unbalanced beliefs, preferences, or prejudices toward certain groups, characteristics, or ideas that influence decision-making processes or assessments within a workplace context. 

At Phenom, we take all three definitions of bias into consideration — so much so that we built the Phenom Fairness and Validity Framework to help us monitor, test, and determine if our AI-powered features are compliant with everything we just highlighted. 

One use case example is our Fit Score feature. To test its adverse impact on the hiring process, we performed a bias audit. The results?

Phenom AI: The Backbone of an Intelligent Talent Experience

AI Day 2023 was full of technical insights into what really goes on in the background. The immense intricacies and overwhelming number of data points, guidelines, infrastructures, and custom-trained models are what make the Phenom platform different from any other HR technology on the market.

Our commitment to helping one billion people find the right work drives our development and product teams to create innovative solutions that help:

• Best-fit candidates find and choose the right job faster

• Employees develop their skills and evolve their careers

• Recruiters become wildly productive

• Managers build stronger-performing teams 

• HR leaders align employee development with company goals through an intelligent workforce planning solution

• HRIS teams seamlessly integrate with your HR tech stack

We pay attention to the details to ensure that each of our customers can feel confident using AI to streamline their everyday activities and create more impactful, positive experiences for individuals in each stage of the talent lifecycle. And with out-of-the-box intelligence, your teams can hit the ground running from day one. 

To discover exactly how Phenom AI can help your TA and TM teams hire, develop, and retain top talent, book a personalized demo with our team today.

*The information provided on this website does not, and is not intended to, constitute legal advice. All information, content, and materials available on this site are for general informational purposes only.