ChatGPT Problem with Counting: Handling Character Limits ChatGPT Responses
The Character Count Dilemma
While HTML forms and websites often place a limit on the number of characters a response can have, AI content generators, ChatGPT included, face a challenge following these constraints. For instance, consider the simple instruction:
- Write a recruiting email to the CANDIDATE about POSITION
- Limit the email to 300 characters
You might expect a response within the set 300 character limit... But ChatGPT can't quite count. The first response was 343 characters, the second ballooned to 700, and the third was higher still at 800 characters! ChatGPT understood that 300 characters meant shorter than usual, it didn't know how short and couldn't follow it!
Regardless of if ChatGPT tried to keep them short, they still don’t fit into an HTML form with a 300-character limit - so from a developer's perspective using the ChatGPT API, the response isn't great.
Iterating Towards a Solution
Although ChatGPT seems to have problems with numbers, it does understand the concept of shortening a message. As a workaround, I tried the following prompt:
Shorten the message to {char_limit} characters or less. Try to keep it personalized and relevant. Remove at least {len(message)+5 - char_limit} characters.
Yet, this approach also has its limitations. Sometimes, it takes multiple tries to get a response within the desired character range, which, in turn, means several API calls. The need to iteratively call the API can add some latency to your application is not the most efficient solution.
You might wonder, why ask the model to remove an extra 5 characters? The reason is simple: ChatGPT's problem with counting. By asking it to remove a bit more than necessary, there's a higher probability of hitting the desired character range in fewer iterations. In essence, removing approximately five characters might equate to removing a word, so the system might then know to eliminate an extra word. It's also worth noting that if you give the exact number of characters to remove, ChatGPT might hover around that limit without making any significant reduction. (Trust me, it was painful trial and error!)
In Conclusion
Taming an AI model like ChatGPT requires a mix of understanding its strengths and navigating its weaknesses. As developers and users, continuous trial and error is part of the journey to optimal results.
On that note, stay tuned for more insights into ChatGPT's number handling! If you've had similar experiences or found better ways to handle character limitations with ChatGPT, I'd love to hear your thoughts and solutions!