Code experiments: A Computer Weekly Downtime Upload podcast
Summary
We speak to GoDaddy CTO Charles Beadnall about how to prove an IT project works as intended There are many who will argue the case for building and deploying a new IT system to streamline a business process. Charles Beadnall, chief technology officer (CTO) at GoDaddy, questions the validity of current software development, in terms of whether a new implentation genuinely makes a difference to what came before it. He says: “You have a hypothesis, you test that hypothesis with a control dataset, then you run the experiment and evaluate the outcomes with an experimental group such that you can compare the outcomes against your hypothesis.” While this idea of running software projects like an experiment seems straightforward, Beadnall says it’s a “bit more difficult in practice because it represents a giant mindset shift for product management and engineering – it’s not something that comes as naturally to how many teams operate today”. But if the team has not collected real data in terms of how long it takes for a particular action to be completed, it is hard to judge whether the project has been a success or not. For Beadnall, artificial intelligence (AI) tooling has reached a point where it is easier than ever to start experimenting with large language models like ChatGPT.