Using Meta’s new LLaMa 2 generative AI tool as a chatbot and comparing it to ChatGPT and Google Bard

Using Meta's new LLaMa 2 generative AI tool as a chatbot and comparing it to ChatGPT and Google Bard

LLaMa 2: A Promising but Unpolished Chatbot

As the name suggests, Meta’s LLaMa 2 (Large Language Model Meta AI) is the second version of their language generation tool. With 40% more training data and double the context length, LLaMa 2 is a powerful open-source model with potential for customization and adaptation. However, compared to existing bots like ChatGPT, Bing Chat, and Google Bard, it falls slightly short in terms of out-of-the-box consumer-facing AI assistance for tasks such as writing or researching.

A Different Kind of AI Bot

LLaMa 2 is not primarily designed to be a chatbot. Instead, it serves as a general LLM (Large Language Model) that developers can download and customize. Meta CEO Mark Zuckerberg intends to improve and advance the model through developer contributions. To use LLaMa 2 as a chatbot, special demo versions like HuggingChat on platforms like Hugging Face are available. HuggingChat is an open-source alternative to ChatGPT, specifically created by the developer community.

According to Philipp Schmid, the technical director of Hugging Face, LLaMa 2 is comparable to other AI bots, but with a unique advantage. It can be shaped inexpensively for specific needs, unlike highly fine-tuned products like Bing Chat. However, it’s important to note that LLaMa 2 does not have internet connectivity, meaning it has a “knowledge cutoff” at December 2022. While the creators of HuggingChat added a web search option, it’s still in the early stages and doesn’t provide the same capacity as other web-searching chatbots like Bing Chat or Google Bard.

In a paper announcing the release of LLaMa 2, Facebook researchers acknowledged that it generally performs better than existing open-source models and closely competes with closed-source models like ChatGPT. However, it doesn’t fully compare to GPT4, OpenAI’s most advanced LLM, yet.

Putting LLaMa 2 to the Test

To assess LLaMa 2’s capabilities, I tried various tasks that generative AI tools are useful for. When asked to write an email to my coworkers about my upcoming out-of-town trip, LLaMa 2 generated a decent memo suitable for the formal corporate space. However, when I asked it to draft an email declining a job offer, the initial response was impersonal. By requesting more specificity, personalization, and apology, LLaMa 2 produced a wordy, formal email that would suffice.

• Rivian raises 2023 production goal, exceeds Q2 revenue expectations...
• US credit card debt exceeds $1 trillion, while overall consumer deb...
• Barrick open to Saudi wealth fund buying stake in Reko Diq project ...

LLaMa 2 has the potential to handle tasks like writing summaries, drafting memos, and politely declining meetings. Compared to ChatGPT, I found LLaMa 2’s penmanship to be decent but overly formal. ChatGPT, on the other hand, often infuses a stronger human-like touch into its language, making it more appealing for these tasks.

However, for more creative or “literary” writing tasks, LLaMa 2’s performance was mixed. It struggled to follow word count instructions and didn’t consistently produce high-quality output. While it could generate a haiku or a 16-line poem on any given topic, the quality was questionable. For example, the haiku it generated, “Circuits hum with life, Processors pace the digital strife, Binary symphony,” may not be considered a strong haiku by literary standards.

When It Comes to Research, LLaMA 2 Isn’t Up to Par

LLaMa 2 showed its limitations when it came to research tasks. While it could provide a summary of the property crisis in China, it wasn’t able to condense the information into a 50-word summary with accurate citations. Additionally, its “Search web” function led to broken links, rendering it ineffective for accessing up-to-date information. With a knowledge cutoff at December 2022 and faulty search capabilities, LLaMa 2 is not suitable for important research purposes.

It’s essential to remember that LLaMa 2 is still in the demo stage and requires fine-tuning. Hallucinated citations and a limited knowledge cutoff are potential pitfalls. As with all generative AI tools, thorough research is necessary to validate the created content. However, tools like LLaMa 2-based HuggingChat hold promise and are constantly being refined and updated. I encourage you to give this bot a try while being mindful of its limitations.

You can try HuggingChat here.

And here’s ANBLE’s 3-step guide on how to use chat AI tools.