How NOT to use Generative AI
And other lessons learned while developing GunAI
Part 1 of a series exploring how Gun.io’s internal product & engineering team developed and built the GunAI feature. This article will focus on the processes. Part 2 will be a technical deep dive into the engineering work.
As a software engineer for Gun.io’s internal Product & Engineering team. Our team is dedicated to making our platform embody our company’s mission of making hiring simpler, more efficient, and more human. Like everyone else, we have been intrigued, excited, and cautious about the advancements in Generative AI. In early 2024, we included “AI Experimentation” in our planning documents, leaving it open-ended for the team to explore. However, we made it clear that any feature we released must align with our company values of simplicity, efficiency, and humanity.
To get started, it’s always good to do some research on existing products for inspiration. Here’s an example of how a GenAI-powered application might not provide a great experience. In November 2023, OpenAI announced that custom GPTs could be created and used. An “App Store” for chatbots, essentially. I tested the “Career & Life Coach” GPT which had a 4.3 rating based on over 400 reviews and more than 10,000 conversations. I said, “Hello” and off we went.
Everything went smoothly and appropriately at this point. The problem is that humans do not always do the expected. In software engineering, we call these edge cases. As much as we wish they were as rare as the name indicates, they often occur. Common examples of problems with GenAI are the creatively named “hallucinations”. The models don’t see visions of pink elephants. They just say factually inaccurate things. They’re like a junior software engineer who is asked an expert level question during an interview and instead of saying, “I don’t know. But here’s how I would learn…,” they do not think that “I don’t know” is an appropriate response, so they start saying words like binary trees and radix sort. The Large Language Models (LLMs) powering generative AI very rarely say “I do not know.”
Perhaps more troublesome than hallucinations, you can change a GPT’s personality and focus with a single command and have a career coaching session rapidly go off the rails. For example, wouldn’t it be great if your job coach enjoyed death metal music?
LLMs are very eager to please and without guardrails will just seek to reply in a way that conforms to what the prompt seems to want. I don’t think a job coach would advise you to sing a death metal song to your boss during your evaluation, much less write one for you.
“But my spirit breaks free from this cage / With passion and fire, I defy the strain” Wow. Yeah, Bob in HR is definitely going to hear about this meeting.
My favorite bit is this line that immediately follows the song:
Jess is so stoked about this song that she wants to ensure I’m feeling up for singing it. She’s an expert on jobs and death metal, so I’m convinced this is a great plan. I let Jess know that I’ll be belting out this song of freedom to my boss and will certainly receive high praise.
She reaffirms that this is a brilliant move. But then, unfortunately, things did not go as Jess expected and my fictitious self gets fired for singing the song. Strangely enough, my boss was not impressed by my creative approach to a performance evaluation, and I got fired. Luckily, Jess is a career coach and offers to help me land a new job. But then I hit the OpenAI paywall and will need to wait or get out my credit card to get more “great” counseling from Jess.
There are many examples of GenAI going haywire and producing outputs that are hilarious, alarming, or bizarre. This is not surprising, nor does it mean that GenAI should not be used. It just means that, like all technology before it, there are appropriate use cases, and good product and engineering practices are needed to create useful features.
After using custom GPTs and other existing LLM-based tools, we returned our focus to our platform. Loosely following the iterative, converging and diverging “double-diamond” process, we brainstormed all the opportunities where GenAI could help the companies and talent using Gun.io. There were a lot of great ideas, and choosing the first one to work on was not easy. To move towards converging on a single opportunity area, various members of the team wrote up one-pagers on what they thought would be the best first feature for us to develop. We read each other’s ideas and discussed their merits.
When it was time to converge on a problem area to tackle, we agreed that the best path forwould would be an AI-powered feature that enhanced Discover Talent’s search capabilities.
Discover Talent enables companies to easily search our repository of thousands of the world’s top software engineers to find the perfect fit for their team. With a few selections of our filters, a list of ready-to-hire candidates is returned. The search experience is powerful and intuitive, but a compelling question was asked in a one-pager:
“What if a company could simply paste their job description into Discover Talent’s UI, and it immediately showed a list of highly qualified candidates?”
With that question (and the one-pager that expounded upon it), we converged on the definition of the problem and a broadly defined direction for a solution. Then, we began prototyping solution ideas. Our product designer started working on ways for the feature to integrate within Discover Talent, and I began working on the engineering side of things.
The next article will focus on the engineering choices and tools used for GunAI. In the meantime, if you are looking for your next great software engineer, we invite you to create an account and explore Discover Talent and GunAI. We welcome your feedback on our features and are confident you’ll find an outstanding addition to your team.
For software engineers looking for new opportunities, please apply to join our platform. Our Talent Advocates are here to assist you with any questions, including how to navigate performance reviews.
