LLM Chat Bot

A simple chat bot made with vercel's Ai toolkit and ollama

Video Preview

demo.mp4

Installation

Install Ollama (LLM backend)

This project relies on Ollama to work. It should be installed on your system.

1. Download Ollama for your operating system

2. Download Meta's Llama 3 model (≈4GB) by running ollama run llama3 in your terminal

   Node: If the responses are slow you may switch to a faster model. Check out the FAQ at the end.

Run Development Version

1. Run pnpm i to install all dependencies
2. Run pnpm dev to start dev server or pnpm build-preview for productioin server
3. Open localhost:3000/ on your browser

Tech

• NextJS
• Typescript
• React
• Vercel's Ai Toolkit
• Ollama
• Meta's llama3 model
• Tailwind
• NextUI

Requirements:

• Single User Assumption: Design the application for a single user; there is no need to implement authentication or a user management system.

• Session Memory: The application must be capable of recalling messages from previous sessions with the user. (use sqlite as database)

• User Feedback Implementation: Incorporate a like/dislike button for each message from the AI. The application should record this feedback in a database and display it alongside the messages in subsequent sessions.

Bonus Task:

• Enhanced Formatting: Implement the capability for the AI to deliver responses in Markdown format.

FAQ

How do I change the LLM model?

You need to download one of many available models from Ollama's library and then make the following changes to /app/api/v1/chat/route.ts

Note: I am using llama3 just as an example

const result = await streamText({
--  model: ollama("llama3"), // Slower but better model
++  model: ollama("phi3"), // Faster but older less capable model
  messages,
})

Why did I use Ollama?

Because it runs locally and is free. Unlinke APIs from OpenAi, Athropic, etc.