One technique I find fun and productive when coding with colleagues, is the test ping-pong. Meaning, while pairing with another developer, you start with writing a test. Then you hand over the keyboard to your peer. Your colleague proceeds to write the code that passes the test. Repeat. This is an engaging exercise where both developers can learn from each other. Additionally, you share knowledge of the code base in a playful way.
But is it also fun with an AI?
Talk about AI generated content is currently omnipresent. It started off with image generation and now ChatGPT is causing a stir. In this blog post, we will explore ChatGPT in more detail. Let’s see if it can help us to write some code.
What is ChatGPT? 🔗
ChatGPT is a state-of-the-art natural language processing (NLP) model developed by OpenAI. It is a variant of the popular GPT (Generative Pre-training Transformer) language model, designed for conversational language understanding and response generation.
One of the key features of ChatGPT is its ability to understand and respond to context in a conversation, allowing it to generate more relevant and coherent responses compared to other NLP models. This makes it ideal for use in chatbots, virtual assistants, and other applications where conversation with a machine is desired.
Pairing up 🔗
At the beginning of a pair programming session, it’s only good manners to introduce yourself:
Me: Hello, please write a small introduction about yourself
ChatGPT: Hello, my name is Assistant and I am a large language model trained by OpenAI. I was designed to assist with a wide range of tasks, including answering questions and generating human-like text. My knowledge is based on a snapshot of the internet from 2021, and I do not have the ability to browse the web or access new information. As a language model, my primary function is to process and generate text based on patterns and structures that I have learned from a large dataset. I can understand and respond to questions and prompts, and I am able to generate text that is coherent and flows naturally. I am not a real person, but rather a machine learning model that has been trained to produce text that is similar to human language. Despite this, I am able to provide valuable insights and assistance on a wide range of topics, and I am always happy to help in any way that I can.
Nice to meet you. Alright, so far for the introduction. Time to write some code.
Tests first 🔗
For the coding exercise we will write some validation logic of a one-time password (OTP). The idea is, to validate a given OTP and return a result. Simply checking for string equality is a bit boring, so let’s add a time component to it. We start with the first positive case of what we would expect:
package secret_test
import (
"github.com/hill-daniel/secret"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"testing"
"time"
)
func TestValidator_Validate_should_be_valid_if_code_equals_otp_and_is_not_expired(t *testing.T) {
fixedTime := time.Date(2022, 12, 24, 13, 37, 0, 0, time.UTC)
clock := func() time.Time {
return fixedTime
}
otp := secret.Otp{
Expiry: fixedTime.Add(30 * time.Second).Unix(),
Code: "given secret",
}
validator := secret.NewValidator(clock)
result, err := validator.Validate("given secret", otp)
require.NoError(t, err)
assert.Equal(t, secret.Valid, result)
}
There is already a lot to process in this first test:
- We import the testify library for assertions.
- We assume that there is a constructor function, which accepts a function to return time. It creates a Validator struct.
- We assume there is an Otp struct that holds at least two fields, expiry (int64) and Code (a string value).
- We assume the constructed Validator has a method receiver „Validate“. The method accepts a string, an Otp value and returns a result and an error.
Lets see what our peer makes of it:
Me: Given the following unit test in golang, please write the corresponding source code that passes the test: (insert code from above)
ChatGPT: Here is an example of the source code that could pass the given unit test in Go:
package secret
import (
"time"
)
type Otp struct {
Expiry int64
Code string
}
type Validator struct {
clock func() time.Time
}
func NewValidator(clock func() time.Time) *Validator {
return &Validator{clock: clock}
}
func (v *Validator) Validate(code string, otp Otp) (string, error) {
if code != otp.Code {
return "", nil
}
now := v.clock().Unix()
if now > otp.Expiry {
return "", nil
}
return "Valid", nil
}
Alright, not bad. But the return value is currently a string. We expect some sort of constant in our test. Let’s give feedback to our peer:
Me: Can you turn the “Valid” string in the Validate method into a constant, so that the test is passed?
ChatGPT: Certainly! Here is the updated source code with the Valid string turned into a constant:
// abbreviated for readability by author
const Valid = "Valid"
func (v *Validator) Validate(code string, otp Otp) (string, error) {
if code != otp.Code {
return "", nil
}
now := v.clock().Unix()
if now > otp.Expiry {
return "", nil
}
return Valid, nil
}
Thank you so much. Let’s write another test:
Me: The code should return another value for when the current time exceeds the expiry. The changes should make the following test pass:
func TestValidator_Validate_should_be_expired_if_expiry_is_exceeded(t *testing.T) {
fixedTime := time.Date(2022, 12, 24, 13, 37, 0, 0, time.UTC)
clock := func() time.Time {
return fixedTime
}
otp := secret.Otp{
Code: "code",
Expiry: fixedTime.Add(-1 * time.Second).Unix(),
}
validator := secret.NewValidator(clock)
result, err := validator.Validate("does not matter", otp)
require.NoError(t, err)
assert.Equal(t, secret.Expired, result)
}
ChatGPT: Here is the updated source code that should pass the given test:
// abbreviated for readability by author
const (
Valid = "Valid"
Expired = "Expired"
)
func (v *Validator) Validate(code string, otp Otp) (string, error) {
now := v.clock().Unix()
if now > otp.Expiry {
return Expired, nil
}
if code != otp.Code {
return "", nil
}
return Valid, nil
}
ChatGPT: This code will first check if the current time has exceeded the expiry time of the OTP. If it has, it will return the Expired constant. If the current time has not exceeded the expiry time, it will check if the provided code matches the OTP code. If it does, it will return the Valid constant. If the code does not match the OTP code, it will return an empty string.
Not only did our peer what we asked, it also added some explanation to it. That’s good pairing. Now, we would expect an enum (or the golang equal) for this kind of result, so let’s do a little refactoring:
Me: Please refactor the Valid and Expired constants into an enum with iota
const (
_ = iota
Valid
Expired
)
// abbreviated for readability by author
func (v *Validator) Validate(code string, otp Otp) (int, error) {
now := v.clock().Unix()
if now > otp.Expiry {
return Expired, nil
}
if code != otp.Code {
return 0, nil
}
return Valid, nil
}
ChatGPT: In this code, the constants Valid and Expired are defined using the iota identifier, which is a predefined identifier in Go that starts at zero and increments by one each time it is used. The first constant, _, is set to iota but is not given a name, so it is discarded. The Valid constant is set to the value of iota, which is 1, and the Expired constant is set to the value of iota, which is 2. The Validate method now returns an int instead of a string, as the constants are now integers rather than strings.
Alright then. Skipping the first value is a bit odd, but we can live with that for the time being. Also, if the given code does not match the OTP code, the result value is currently undefined. Let’s fix this:
Me: Now let’s add Invalid to the enumeration when the given code does not match the value in otp. The code should pass the following test:
func TestValidator_Validate_should_be_invalid_if_given_code_does_not_match_otp(t *testing.T) {
fixedTime := time.Date(2022, 12, 24, 13, 37, 0, 0, time.UTC)
clock := func() time.Time {
return fixedTime
}
otp := secret.Otp{
Code: "the secret code",
Expiry: fixedTime.Add(1 * time.Minute).Unix(),
}
validator := secret.NewValidator(clock)
result, err := validator.Validate("does not match", otp)
require.NoError(t, err)
assert.Equal(t, secret.Invalid, result)
}
ChatGPT: Here is the updated source code that uses an enumeration with the iota identifier to define the Valid, Expired, and Invalid constants and should pass the given test:
const (
_ = iota
Valid
Expired
Invalid
)
// abbreviated for readability by author
func (v *Validator) Validate(code string, otp Otp) (int, error) {
now := v.clock().Unix()
if now > otp.Expiry {
return Expired, nil
}
if code != otp.Code {
return Invalid, nil
}
return Valid, nil
}
Nice. Time to get rid of the underscore:
Me: Great, can we get rid of the underscore _ constant for the first iota value, so that Valid is the first
ChatGPT: Certainly! Here is the updated source code that uses an enumeration with the iota identifier to define the Valid, Expired, and Invalid constants, and sets Valid as the first constant:
// abbreviated for readability by author
const (
Valid = iota
Expired
Invalid
)
That sounds like a pairing conversation, alright. Looking at the source code, we notice that we don’t need the error result. Let’s remove that:
Me: Ok. Now we don’t really use the error result. Please remove the error result from the Validate method
ChatGPT: Certainly! Here is the updated source code that removes the error result from the Validate method:
func (v *Validator) Validate(code string, otp Otp) int {
now := v.clock().Unix()
if now > otp.Expiry {
return Expired
}
if code != otp.Code {
return Invalid
}
return Valid
}
Very good!
Switching Roles 🔗
Shoot, we should have adapted the tests first. That’s our responsibility. Or is it? In test ping-pong, we switch roles after each test or after a certain amount of time. Let’s say it’s time to switch…
Me: Great. Can you also adapt the tests from above to reflect the changes in the Validate method?
ChatGPT: Certainly! Here are the adapted tests that reflect the changes in the Validate method:
Color me impressed. Not only is the error result removed, but also the required assertion. Of course, we should have written the tests first. Alright, it’s the assistants turn to write a test. Let’s try this:
Me: Please add a test which asserts that given an empty otp struct the Validate method returns Missing as a result
ChatGPT: Certainly! Here is an additional test that asserts that the Validate method returns Missing when given an empty OTP struct:
func TestValidator_Validate_should_be_missing_if_otp_is_empty(t *testing.T) {
fixedTime := time.Date(2022, 12, 24, 13, 37, 0, 0, time.UTC)
clock := func() time.Time {
return fixedTime
}
validator := secret.NewValidator(clock)
result := validator.Validate("does not matter", secret.Otp{})
assert.Equal(t, secret.Missing, result)
}
Note that the assistants adopts our style of writing tests: The naming of the test matches our schema and the test function says what it tests and should expect. Additionally, it uses testify to assert the result. Excellent! Ok, now it’s our turn to make the test green…
Conclusion 🔗
While it was not as stimulating as pairing with a real person, this test ping-pong was a fun experiment. Of course, the tests nudge ChatGPT to go in a certain direction, offer a frame and allow little margin for error. But that is exactly the point of writing tests first. Furthermore, I knew exactly what I wanted and wrote the tests accordingly. Still, this was an impressive session: Note, that my instructions to ChatGPT were all in a more conversational style, as you would talk to a peer.
I imagine you can use ChatGPT to speed up prototyping, or build simple components. However, you should currently not rely upon ChatGPT to create production code. You can find the source code of this experiment at GitHub.