The last week or so I’ve been working on porting over my Python framework I conceived in writing a wrapper around Google Agent ADK to Go. It’s been a frustrating experience due to the differences in languages and libraries. While both have a similar setup, there are enough nuances that make a 1:1 port impossible. A simple thing is where Python might use async/await while Go has no need. On the other hand, dealing with a heavily typed, interface and composition driven language presents its own challenges. Someone suggested that I blog about my experiences and here we are.
For the framework I came up with in Python, the idea was to use the command design pattern in creating a format that allows one to manage the Google Agent ADK’s lifecycle for an agent. I combined that design pattern with a template pattern style in a process/execute style function that makes calls to invoke various key moments in the lifecycle of the agent chain. However, with the Python implementation, I use a combination of their version of an interface with abstract classes and inheritance so that my subclasses can implement the key virtual methods/callbacks that are used to handle the specific processing logic such as defining the template, understanding how to map a parameter object into the prompt data or managing the output that gets returned from the ADK’s runner.
Go does not have inheritance and uses composition along with interfaces. So I had to learn how to deal with the equivalent of “inherited” methods that need to be overloaded in the subclasses. But once I had a basic setup working, I was able to get a simple agent working. The real first challenge came in implementing the complex tooling portion. For the ADK and GenAI, tools are functions that the AI can invoke to help perform its work. Along those lines, you can force an agent to obey a schema for input/output. The output is very important because you can get an unstructured string which is harder to manage than say JSON. In the case of Python, you generally use something like Pylon to define schemas. However, I found the schemas to be painful to work with since Python has more implied types. Go is easier to manage since it’s typed and uses a struct type of object to define a schema.
Where my recent problem occurred is a problem I saw in the past where an OutputSchema would cause an issue when run side by side with a tool. However, supposedly the issue was resolved. What I read though is that only certain models support OutputSchema with tools. So to get that to work, you may have to run a subagent. That’s where I would get jammed with Google Search Tool since it’s meant to be a “native” tool and does not support a function tool that you’d normally write. So I would get jammed there because I’m avoiding entering my billing info, which severely limits my ability to interact with models (as well as giving me a very small number of requests per day).
Along the way, I made a decision to try Ollama. I installed it to a Docker instance that runs in the same container as my project. However, it’s painfully slow and very resource consumptive. I had to downgrade the model and bump up the ram to be 12GB. In addition, when a simple query is run, the CPU spikes to near 90%, which is ridiculous. I did manage to get a response hooked in, using the Google Agent ADK along with a simple wrapper to hook in an Ollama model. But the response time would take at least a minute before getting any useful information. Worse yet the model wasn’t trained the same way Gemini is trained and gave me bad results or suggested I use IMDB for the sample agent I developed. I did try a different approach where I had Gemini give me a prompt and used Ollama to format the response in a direct query (no code involved here). The results were super slow even though it was what I wanted. But I thought a child in the 1st grade could probably do the same thing faster.
Yet in that situation I realized that if I want to pursue using Ollama itself, the pattern there would be me probably using Google Search as a query and having Ollama take the results and structure it. But the speed and resource usage would be the main issues, even if it was me developing locally. I can’t imagine what a giant result set might look like. I think the only way something like this could run reasonably would be to have a dedicated beefy server with tons of CPU/GPUs, memory and HD space.
The real question though is whether or not having something like Ollama running locally is worth it at all. As I mentioned, for the amount of effort vs the potential cost of getting something like that working for this situation, I think it’s no bueno. I’m not even convinced Ollama is a good model to do what I intended. If anything I do realize just why these companies are demanding so much power to run these models if they aren’t turning towards an established model provider.
Instead, I ended up reverting the code to use Gemini again and temporarily removing the search option. I think at least for the Go side of the project, the Google Search Tool might be infeasible unless I can figure out how to wrap it. I think the native tool isn’t good unless you’re using the right Gemini model. I have seen alternate sources for supplying search results at least in terms of Python libraries. So I might have to investigate using one of those. For the thing I need to do though, I don’t think the existing Go tool is a good idea.
But the other problem I’m seeing is that this whole AI race is going to eventually create two clear camps: the have’s and have not’s. You either have the resources (money/power/backing) to develop the tools, the computing power and/or budget to use these existing models or you’ll be jammed because these companies aren’t generous with their offerings. Now, I can’t blame them for putting up a paywall but I wish there was a better middle ground to allow hobbyists or people working on prototypes to use their models with higher rate limits. I think if the results from these AI models were better, I’d be more willing to pay. However, at least in my experience, these models tend to work 80% of the time with regards to accuracy or even availability.
Part of the problem I see is that blindly giving away ones billing info permits these companies to charge you even if you screw up on accident or need to re-run your application as you’re developing it. They don’t have a good sandbox type of mode that can give you reasonable results but not cost you an arm and a leg. Also, I think some of these models get a bad reputation which I think will make adoption much harder.
Another thing I’m beginning to think is that Python is probably the way to go in building AI enabled apps. At least from the tooling I’ve seen because of how much more support it gets (at least for these Google ADK). I think think Go might be the better language to use down the line just because of power and speed. I would like to optimize more around those things rather than dealing with the issues of an interpreted language in production. But what I really want to avoid is having to develop in three separate languages where I’m juggling between Go, Python and Typescript in building things like this.
Leave a Reply
You must be logged in to post a comment.