By Hitarth Koshiya 
Seeking the assistance of an AI partner like Pythagora, who can auto-generate your applications, thereby eliminating the need for you to write any code, is what we are going to explore today. During this blog, we will construct a benchmarking application in which we will test various large language models (LLMs) using a series of queries. Pythagora will empower us to develop a strong application within a single color accurately without writing even a single primary code by hand.
Getting Started with Pythagora
Before we start the building process, see that you have all the tools equipped. Node.js, MongoDB, and the Pythagora plugin for Visual Studio Code (VS Code) are the requirements. Lets begin building the app after you completed these steps.
To start, we are using Pythagora to launch VS Code and we pick the creating of new app option. The initial prompt will be needing the project name, and we are going to name it “Benchmark.” This software will empower people to develop LLMs test, run it, and the final report will be sent to the public.
Defining the Application
Your next task is to meticulously describe our application in detail. The task is very important, as Pythagora uses this information to know the application’s functioning. The following will be a short summary of the description we choose to use:
With this detailed prompt entered, we hit the send button to initiate the creation process. Pythagora will take care of the backend processes, allowing us to focus on higher-level tasks:
- A homepage having a welcome message and a display of the magazines that are being out.
- Users should be able to click on a test to view its details.
- User authentication, along with administrator roles.
- A dashboard for the administration to manage both the tests and the users.
- The creation of a new test page.
The Building Process
Pythagora uses a variety of agents to handle the development phase. The spec writer appraises the intricacy of our prompt, and the architect agent bootstraps the application. In our case, we will employ the Node.js and Express framework besides the MongoDB for database.
After the architecture is put together, Pythagora will notify us and ask us for the entering of our API keys. Those of OpenAI and Anthropic are the keys that will be necessary for us to be able to use their APIs in order to perform the tests.
Iterative Development with Pythagora
Pythagora will be regulating the growth cycle during the development process by continuously asking for input. It will generate code snippets, update models, and create routes automatically. In-Parallel, it will give the user model a role field, and hence, we can always depend on its reliability.
Pythagora allows users to see the project’s growth with logs and updates. It offers assistance on how to debug it productively if there is any issue. This practice is recurrent and improves the developer’s knowledge overall.
Setting Up Authentication
One of the first major tasks is to implement user authentication. Pythagora will write scripts to manage user registration and roles. We can take a trial for registration process to see if everything works as per our requirement.
After a user is created on our platform, we can now confirm in the MongoDB database that the document regarding the user information has been created correctly and it includes a hashed password and a role.
Admin Dashboard and User Roles
Following the user authentication, the next step is to come up with an admin dashboard. The creation of this dashboard will give administrators the opportunity to successfully handle users and their roles. Pythagora generates the code required, which we then verify through test runs to ensure that the dashboard is functioning correctly.
Once we have the admin dashboard, we can also introduce a feature for changing users from viewers to creators and vice versa. This function is vital to supervise the permissions in the application.
Creating the Test Management Interface
Further , we will also focus on telling where the application of open source technology can take you. In this article, as the introduction to it, we will make a general description of this engineering construct. The analyzing of that model is the subject of the next section of our work. My work consists of the last section.
Pythagora helps us use the pagination for the test list view in this way that the users can browse through several tests without any difficulty. We can further insert into the database some test data for testing and debugging purposes.
Dynamic Test Creation
The most attractive feature of our application is the dynamic test-creation form. Here, it allows users to specify test parameters, such as user messages and review messages, and the number of requests. Furthermore, a user can add many LLM providers dynamically.
Pythagora writes the code for these forms and also gives real-time feedback so that everything works as expected.
Backend Functionality and Testing
We wish to start laying out our frontend but, crucially in the meantime, we must also design the backend logic for creating and running tests. All of this Pythagora takes care of in a pretty abstract way by automatically writing and refining the backend code along with incremental testing of each part to create the robust result.
Now that we have the backend working, it is quite trivial to include publishing functionality for tests and allow users to share their results with other users by marking some tests as viewable by everyone.
Real-Time Progress Tracking
As for test executions, we also included real-time progress tracking to further improve user experience. Therefore, it updates the users about the statuses of their tests, hence bringing about greater transparency and usability. We test this feature gradually to be as sure as possible that progress bars and status updates bring forward correct ongoing procedures.
Debugging and Iteration
This will walk through every step of the development cycle and still encounter various problem areas that require debugging. Pythagora makes the approach smooth for us in that we can articulate issues; it thus produces log traces that will help identify the problems. In development, we improve our application after receiving feedbacks and tests on it.
Final Touches and Deployment
As we approach the final stages of development, we update the homepage to display a list of published tests. Once everything is in place, we can deploy the application with just one click. Pythagora handles the deployment process, making it easy to publish our application online.
Conclusion
Within two hours, we build a full stack LLM benchmarking application with Pythagora. This is so powerful in creating complex applications that you don’t even have to write a single line of code yourself. We have used these features: User authentication, admin dashboard, dynamic test creation, and real-time tracking.
This has been illustrated in the most significant development journey, where Pythagora has demonstrated enormous potential in simplifying and accelerating the process for application development, using his AI capabilities. If you’d like to take a look at an opportunity to use Pythagora on your next project, be sure to follow along with some of the resources linked below!
