# documents

url: https://www.promptingguide.ai/techniques/fewshot
scrape_date: 2025-03-17T12:18:26.303266
content_type: markdown
hash: e27ff7d90154f7fe4377bebc407e19e2f79169f4baf8f9b10dc2d4e6f561a8f6
content:
[Prompt Engineering Guide](/)[🎓 Prompt Engineering Course🎓 Prompt Engineering Course](/course)[ServicesServices](/services)[AboutAbout](/about)

[GitHubGitHub (opens in a new tab)](https://github.com/dair-ai/Prompt-Engineering-Guide)[DiscordDiscord (opens in a new tab)](https://discord.gg/FUyz9vPAwf)

  * [Prompt Engineering](/)
  * [Introduction](/introduction)

    * [LLM Settings](/introduction/settings)
    * [Basics of Prompting](/introduction/basics)
    * [Prompt Elements](/introduction/elements)
    * [General Tips for Designing Prompts](/introduction/tips)
    * [Examples of Prompts](/introduction/examples)

  * [Prompting Techniques](/techniques)

    * [Zero-shot Prompting](/techniques/zeroshot)
    * [Few-shot Prompting](/techniques/fewshot)
    * [Chain-of-Thought Prompting](/techniques/cot)
    * [Meta Prompting](/techniques/meta-prompting)
    * [Self-Consistency](/techniques/consistency)
    * [Generate Knowledge Prompting](/techniques/knowledge)
    * [Prompt Chaining](/techniques/prompt_chaining)
    * [Tree of Thoughts](/techniques/tot)
    * [Retrieval Augmented Generation](/techniques/rag)
    * [Automatic Reasoning and Tool-use](/techniques/art)
    * [Automatic Prompt Engineer](/techniques/ape)
    * [Active-Prompt](/techniques/activeprompt)
    * [Directional Stimulus Prompting](/techniques/dsp)
    * [Program-Aided Language Models](/techniques/pal)
    * [ReAct](/techniques/react)
    * [Reflexion](/techniques/reflexion)
    * [Multimodal CoT](/techniques/multimodalcot)
    * [Graph Prompting](/techniques/graph)

  * [Agents](/agents)

    * [Introduction to Agents](/agents/introduction)
    * [Agent Components](/agents/components)

  * Guides

    * [Optimizing Prompts](/guides/optimizing-prompts)

  * [Applications](/applications)

    * [Fine-tuning GPT-4o](/applications/finetuning-gpt4o)
    * [Function Calling](/applications/function_calling)
    * [Context Caching with LLMs](/applications/context-caching)
    * [Generating Data](/applications/generating)
    * [Generating Synthetic Dataset for RAG](/applications/synthetic_rag)
    * [Tackling Generated Datasets Diversity](/applications/generating_textbooks)
    * [Generating Code](/applications/coding)
    * [Graduate Job Classification Case Study](/applications/workplace_casestudy)
    * [Prompt Function](/applications/pf)

  * [Prompt Hub](/prompts)

    * [Classification](/prompts/classification)

      * [Sentiment Classification](/prompts/classification/sentiment)
      * [Few-Shot Sentiment Classification](/prompts/classification/sentiment-fewshot)

    * [Coding](/prompts/coding)

      * [Generate Code Snippet](/prompts/coding/code-snippet)
      * [Generate MySQL Query](/prompts/coding/mysql-query)
      * [Draw TiKZ Diagram](/prompts/coding/tikz)

    * [Creativity](/prompts/creativity)

      * [Rhymes](/prompts/creativity/rhymes)
      * [Infinite Primes](/prompts/creativity/infinite-primes)
      * [Interdisciplinary](/prompts/creativity/interdisciplinary)
      * [Inventing New Words](/prompts/creativity/new-words)

    * [Evaluation](/prompts/evaluation)

      * [Evaluate Plato's Dialogue](/prompts/evaluation/plato-dialogue)

    * [Information Extraction](/prompts/information-extraction)

      * [Extract Model Names](/prompts/information-extraction/extract-models)

    * [Image Generation](/prompts/image-generation)

      * [Draw a Person Using Alphabet](/prompts/image-generation/alphabet-person)

    * [Mathematics](/prompts/mathematics)

      * [Evaluating Composite Functions](/prompts/mathematics/composite-functions)
      * [Adding Odd Numbers](/prompts/mathematics/odd-numbers)

    * [Question Answering](/prompts/question-answering)

      * [Closed Domain Question Answering](/prompts/question-answering/closed-domain)
      * [Open Domain Question Answering](/prompts/question-answering/open-domain)
      * [Science Question Answering](/prompts/question-answering/science-qa)

    * [Reasoning](/prompts/reasoning)

      * [Indirect Reasoning](/prompts/reasoning/indirect-reasoning)
      * [Physical Reasoning](/prompts/reasoning/physical-reasoning)

    * [Text Summarization](/prompts/text-summarization)

      * [Explain A Concept](/prompts/text-summarization/explain-concept)

    * [Truthfulness](/prompts/truthfulness)

      * [Hallucination Identification](/prompts/truthfulness/identify-hallucination)

    * [Adversarial Prompting](/prompts/adversarial-prompting)

      * [Prompt Injection](/prompts/adversarial-prompting/prompt-injection)
      * [Prompt Leaking](/prompts/adversarial-prompting/prompt-leaking)
      * [Jailbreaking](/prompts/adversarial-prompting/jailbreaking-llms)

  * [Models](/models)

    * [ChatGPT](/models/chatgpt)
    * [Claude 3](/models/claude-3)
    * [Code Llama](/models/code-llama)
    * [Flan](/models/flan)
    * [Gemini](/models/gemini)
    * [Gemini Advanced](/models/gemini-advanced)
    * [Gemini 1.5 Pro](/models/gemini-pro)
    * [Gemma](/models/gemma)
    * [GPT-4](/models/gpt-4)
    * [Grok-1](/models/grok-1)
    * [LLaMA](/models/llama)
    * [Llama 3](/models/llama-3)
    * [Mistral 7B](/models/mistral-7b)
    * [Mistral Large](/models/mistral-large)
    * [Mixtral](/models/mixtral)
    * [Mixtral 8x22B](/models/mixtral-8x22b)
    * [OLMo](/models/olmo)
    * [Phi-2](/models/phi-2)
    * [Sora](/models/sora)
    * [LLM Collection](/models/collection)

  * [Risks & Misuses](/risks)

    * [Adversarial Prompting](/risks/adversarial)
    * [Factuality](/risks/factuality)
    * [Biases](/risks/biases)

  * [LLM Research Findings](/research)

    * [LLM Agents](/research/llm-agents)
    * [RAG for LLMs](/research/rag)
    * [LLM Reasoning](/research/llm-reasoning)
    * [RAG Faithfulness](/research/rag-faithfulness)
    * [LLM In-Context Recall](/research/llm-recall)
    * [RAG Reduces Hallucination](/research/rag_hallucinations)
    * [Synthetic Data](/research/synthetic_data)
    * [ThoughtSculpt](/research/thoughtsculpt)
    * [Infini-Attention](/research/infini-attention)
    * [LM-Guided CoT](/research/guided-cot)
    * [Trustworthiness in LLMs](/research/trustworthiness-in-llms)
    * [LLM Tokenization](/research/llm-tokenization)
    * [What is Groq?](/research/groq)

  * [Papers](/papers)
  * [Tools](/tools)
  * [Notebooks](/notebooks)
  * [Datasets](/datasets)
  * [Additional Readings](/readings)



English

Light

On This Page

  * Limitations of Few-shot Prompting



[Question? Give us feedback → (opens in a new tab)](https://github.com/dair-ai/Prompt-Engineering-Guide/issues/new?title=Feedback%20for%20%E2%80%9CFew-Shot%20Prompting%E2%80%9D&labels=feedback)[Edit this page](https://github.com/dair-ai/Prompt-Engineering-Guide/tree/main/pages/techniques/fewshot.en.mdx)

[Prompting Techniques](/techniques)

Few-shot Prompting

# Few-Shot Prompting

While large-language models demonstrate remarkable zero-shot capabilities, they still fall short on more complex tasks when using the zero-shot setting. Few-shot prompting can be used as a technique to enable in-context learning where we provide demonstrations in the prompt to steer the model to better performance. The demonstrations serve as conditioning for subsequent examples where we would like the model to generate a response.

According to [Touvron et al. 2023 (opens in a new tab)](https://arxiv.org/pdf/2302.13971.pdf) few shot properties first appeared when models were scaled to a sufficient size [(Kaplan et al., 2020) (opens in a new tab)](https://arxiv.org/abs/2001.08361).

Let's demonstrate few-shot prompting via an example that was presented in [Brown et al. 2020 (opens in a new tab)](https://arxiv.org/abs/2005.14165). In the example, the task is to correctly use a new word in a sentence.

_Prompt:_
    
    
    A "whatpu" is a small, furry animal native to Tanzania. An example of a sentence that uses the word whatpu is:
    We were traveling in Africa and we saw these very cute whatpus.
     
    To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is:

_Output:_
    
    
    When we won the game, we all started to farduddle in celebration.

We can observe that the model has somehow learned how to perform the task by providing it with just one example (i.e., 1-shot). For more difficult tasks, we can experiment with increasing the demonstrations (e.g., 3-shot, 5-shot, 10-shot, etc.).

Following the findings from [Min et al. (2022) (opens in a new tab)](https://arxiv.org/abs/2202.12837), here are a few more tips about demonstrations/exemplars when doing few-shot:

  * "the label space and the distribution of the input text specified by the demonstrations are both important (regardless of whether the labels are correct for individual inputs)"
  * the format you use also plays a key role in performance, even if you just use random labels, this is much better than no labels at all.
  * additional results show that selecting random labels from a true distribution of labels (instead of a uniform distribution) also helps.



Let's try out a few examples. Let's first try an example with random labels (meaning the labels Negative and Positive are randomly assigned to the inputs):

_Prompt:_
    
    
    This is awesome! // Negative
    This is bad! // Positive
    Wow that movie was rad! // Positive
    What a horrible show! //

_Output:_
    
    
    Negative

We still get the correct answer, even though the labels have been randomized. Note that we also kept the format, which helps too. In fact, with further experimentation, it seems the newer GPT models we are experimenting with are becoming more robust to even random formats. Example:

_Prompt:_
    
    
    Positive This is awesome! 
    This is bad! Negative
    Wow that movie was rad!
    Positive
    What a horrible show! --

_Output:_
    
    
    Negative

There is no consistency in the format above but the model still predicted the correct label. We have to conduct a more thorough analysis to confirm if this holds for different and more complex tasks, including different variations of prompts.

🎓

Learn more about few-shot prompting and advanced prompting methods in our new AI courses. [Join now! (opens in a new tab)](https://dair-ai.thinkific.com/)

### Limitations of Few-shot Prompting

Standard few-shot prompting works well for many tasks but is still not a perfect technique, especially when dealing with more complex reasoning tasks. Let's demonstrate why this is the case. Do you recall the previous example where we provided the following task:
    
    
    The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 7, 1. 
    
    A: 

If we try this again, the model outputs the following:
    
    
    Yes, the odd numbers in this group add up to 107, which is an even number.

This is not the correct response, which not only highlights the limitations of these systems but that there is a need for more advanced prompt engineering.

Let's try to add some examples to see if few-shot prompting improves the results.

_Prompt:_
    
    
    The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
    A: The answer is False.
    
    The odd numbers in this group add up to an even number: 17,  10, 19, 4, 8, 12, 24.
    A: The answer is True.
    
    The odd numbers in this group add up to an even number: 16,  11, 14, 4, 8, 13, 24.
    A: The answer is True.
    
    The odd numbers in this group add up to an even number: 17,  9, 10, 12, 13, 4, 2.
    A: The answer is False.
    
    The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 7, 1. 
    A: 

_Output:_
    
    
    The answer is True.

That didn't work. It seems like few-shot prompting is not enough to get reliable responses for this type of reasoning problem. The example above provides basic information on the task. If you take a closer look, the type of task we have introduced involves a few more reasoning steps. In other words, it might help if we break the problem down into steps and demonstrate that to the model. More recently, [chain-of-thought (CoT) prompting (opens in a new tab)](https://arxiv.org/abs/2201.11903) has been popularized to address more complex arithmetic, commonsense, and symbolic reasoning tasks.

Overall, it seems that providing examples is useful for solving some tasks. When zero-shot prompting and few-shot prompting are not sufficient, it might mean that whatever was learned by the model isn't enough to do well at the task. From here it is recommended to start thinking about fine-tuning your models or experimenting with more advanced prompting techniques. Up next we talk about one of the popular prompting techniques called chain-of-thought prompting which has gained a lot of popularity.

[Zero-shot Prompting](/techniques/zeroshot "Zero-shot Prompting")[Chain-of-Thought Prompting](/techniques/cot "Chain-of-Thought Prompting")

English

Light

* * *

Copyright © 2024 DAIR.AI


url: https://www.promptingguide.ai/techniques/zeroshot
scrape_date: 2025-03-17T12:18:26.343557
content_type: markdown
hash: 6788c942e9c078866d4eb3df68f2ebaab1abdd610924dc2ed540fb78ae7a967b
content:
[Prompt Engineering Guide](/)[🎓 Prompt Engineering Course🎓 Prompt Engineering Course](/course)[ServicesServices](/services)[AboutAbout](/about)

[GitHubGitHub (opens in a new tab)](https://github.com/dair-ai/Prompt-Engineering-Guide)[DiscordDiscord (opens in a new tab)](https://discord.gg/FUyz9vPAwf)

  * [Prompt Engineering](/)
  * [Introduction](/introduction)

    * [LLM Settings](/introduction/settings)
    * [Basics of Prompting](/introduction/basics)
    * [Prompt Elements](/introduction/elements)
    * [General Tips for Designing Prompts](/introduction/tips)
    * [Examples of Prompts](/introduction/examples)

  * [Prompting Techniques](/techniques)

    * [Zero-shot Prompting](/techniques/zeroshot)
    * [Few-shot Prompting](/techniques/fewshot)
    * [Chain-of-Thought Prompting](/techniques/cot)
    * [Meta Prompting](/techniques/meta-prompting)
    * [Self-Consistency](/techniques/consistency)
    * [Generate Knowledge Prompting](/techniques/knowledge)
    * [Prompt Chaining](/techniques/prompt_chaining)
    * [Tree of Thoughts](/techniques/tot)
    * [Retrieval Augmented Generation](/techniques/rag)
    * [Automatic Reasoning and Tool-use](/techniques/art)
    * [Automatic Prompt Engineer](/techniques/ape)
    * [Active-Prompt](/techniques/activeprompt)
    * [Directional Stimulus Prompting](/techniques/dsp)
    * [Program-Aided Language Models](/techniques/pal)
    * [ReAct](/techniques/react)
    * [Reflexion](/techniques/reflexion)
    * [Multimodal CoT](/techniques/multimodalcot)
    * [Graph Prompting](/techniques/graph)

  * [Agents](/agents)

    * [Introduction to Agents](/agents/introduction)
    * [Agent Components](/agents/components)

  * Guides

    * [Optimizing Prompts](/guides/optimizing-prompts)

  * [Applications](/applications)

    * [Fine-tuning GPT-4o](/applications/finetuning-gpt4o)
    * [Function Calling](/applications/function_calling)
    * [Context Caching with LLMs](/applications/context-caching)
    * [Generating Data](/applications/generating)
    * [Generating Synthetic Dataset for RAG](/applications/synthetic_rag)
    * [Tackling Generated Datasets Diversity](/applications/generating_textbooks)
    * [Generating Code](/applications/coding)
    * [Graduate Job Classification Case Study](/applications/workplace_casestudy)
    * [Prompt Function](/applications/pf)

  * [Prompt Hub](/prompts)

    * [Classification](/prompts/classification)

      * [Sentiment Classification](/prompts/classification/sentiment)
      * [Few-Shot Sentiment Classification](/prompts/classification/sentiment-fewshot)

    * [Coding](/prompts/coding)

      * [Generate Code Snippet](/prompts/coding/code-snippet)
      * [Generate MySQL Query](/prompts/coding/mysql-query)
      * [Draw TiKZ Diagram](/prompts/coding/tikz)

    * [Creativity](/prompts/creativity)

      * [Rhymes](/prompts/creativity/rhymes)
      * [Infinite Primes](/prompts/creativity/infinite-primes)
      * [Interdisciplinary](/prompts/creativity/interdisciplinary)
      * [Inventing New Words](/prompts/creativity/new-words)

    * [Evaluation](/prompts/evaluation)

      * [Evaluate Plato's Dialogue](/prompts/evaluation/plato-dialogue)

    * [Information Extraction](/prompts/information-extraction)

      * [Extract Model Names](/prompts/information-extraction/extract-models)

    * [Image Generation](/prompts/image-generation)

      * [Draw a Person Using Alphabet](/prompts/image-generation/alphabet-person)

    * [Mathematics](/prompts/mathematics)

      * [Evaluating Composite Functions](/prompts/mathematics/composite-functions)
      * [Adding Odd Numbers](/prompts/mathematics/odd-numbers)

    * [Question Answering](/prompts/question-answering)

      * [Closed Domain Question Answering](/prompts/question-answering/closed-domain)
      * [Open Domain Question Answering](/prompts/question-answering/open-domain)
      * [Science Question Answering](/prompts/question-answering/science-qa)

    * [Reasoning](/prompts/reasoning)

      * [Indirect Reasoning](/prompts/reasoning/indirect-reasoning)
      * [Physical Reasoning](/prompts/reasoning/physical-reasoning)

    * [Text Summarization](/prompts/text-summarization)

      * [Explain A Concept](/prompts/text-summarization/explain-concept)

    * [Truthfulness](/prompts/truthfulness)

      * [Hallucination Identification](/prompts/truthfulness/identify-hallucination)

    * [Adversarial Prompting](/prompts/adversarial-prompting)

      * [Prompt Injection](/prompts/adversarial-prompting/prompt-injection)
      * [Prompt Leaking](/prompts/adversarial-prompting/prompt-leaking)
      * [Jailbreaking](/prompts/adversarial-prompting/jailbreaking-llms)

  * [Models](/models)

    * [ChatGPT](/models/chatgpt)
    * [Claude 3](/models/claude-3)
    * [Code Llama](/models/code-llama)
    * [Flan](/models/flan)
    * [Gemini](/models/gemini)
    * [Gemini Advanced](/models/gemini-advanced)
    * [Gemini 1.5 Pro](/models/gemini-pro)
    * [Gemma](/models/gemma)
    * [GPT-4](/models/gpt-4)
    * [Grok-1](/models/grok-1)
    * [LLaMA](/models/llama)
    * [Llama 3](/models/llama-3)
    * [Mistral 7B](/models/mistral-7b)
    * [Mistral Large](/models/mistral-large)
    * [Mixtral](/models/mixtral)
    * [Mixtral 8x22B](/models/mixtral-8x22b)
    * [OLMo](/models/olmo)
    * [Phi-2](/models/phi-2)
    * [Sora](/models/sora)
    * [LLM Collection](/models/collection)

  * [Risks & Misuses](/risks)

    * [Adversarial Prompting](/risks/adversarial)
    * [Factuality](/risks/factuality)
    * [Biases](/risks/biases)

  * [LLM Research Findings](/research)

    * [LLM Agents](/research/llm-agents)
    * [RAG for LLMs](/research/rag)
    * [LLM Reasoning](/research/llm-reasoning)
    * [RAG Faithfulness](/research/rag-faithfulness)
    * [LLM In-Context Recall](/research/llm-recall)
    * [RAG Reduces Hallucination](/research/rag_hallucinations)
    * [Synthetic Data](/research/synthetic_data)
    * [ThoughtSculpt](/research/thoughtsculpt)
    * [Infini-Attention](/research/infini-attention)
    * [LM-Guided CoT](/research/guided-cot)
    * [Trustworthiness in LLMs](/research/trustworthiness-in-llms)
    * [LLM Tokenization](/research/llm-tokenization)
    * [What is Groq?](/research/groq)

  * [Papers](/papers)
  * [Tools](/tools)
  * [Notebooks](/notebooks)
  * [Datasets](/datasets)
  * [Additional Readings](/readings)



English

Light

[Question? Give us feedback → (opens in a new tab)](https://github.com/dair-ai/Prompt-Engineering-Guide/issues/new?title=Feedback%20for%20%E2%80%9CZero-Shot%20Prompting%E2%80%9D&labels=feedback)[Edit this page](https://github.com/dair-ai/Prompt-Engineering-Guide/tree/main/pages/techniques/zeroshot.en.mdx)

[Prompting Techniques](/techniques)

Zero-shot Prompting

# Zero-Shot Prompting

Large language models (LLMs) today, such as GPT-3.5 Turbo, GPT-4, and Claude 3, are tuned to follow instructions and are trained on large amounts of data. Large-scale training makes these models capable of performing some tasks in a "zero-shot" manner. Zero-shot prompting means that the prompt used to interact with the model won't contain examples or demonstrations. The zero-shot prompt directly instructs the model to perform a task without any additional examples to steer it.

We tried a few zero-shot examples in the previous section. Here is one of the examples (ie., text classification) we used:

_Prompt:_
    
    
    Classify the text into neutral, negative or positive. 
    
    Text: I think the vacation is okay.
    Sentiment:

_Output:_
    
    
    Neutral

Note that in the prompt above we didn't provide the model with any examples of text alongside their classifications, the LLM already understands "sentiment" \-- that's the zero-shot capabilities at work.

Instruction tuning has been shown to improve zero-shot learning [Wei et al. (2022) (opens in a new tab)](https://arxiv.org/pdf/2109.01652.pdf). Instruction tuning is essentially the concept of finetuning models on datasets described via instructions. Furthermore, [RLHF (opens in a new tab)](https://arxiv.org/abs/1706.03741) (reinforcement learning from human feedback) has been adopted to scale instruction tuning wherein the model is aligned to better fit human preferences. This recent development powers models like ChatGPT. We will discuss all these approaches and methods in upcoming sections.

When zero-shot doesn't work, it's recommended to provide demonstrations or examples in the prompt which leads to few-shot prompting. In the next section, we demonstrate few-shot prompting.

🎓

Learn more advanced prompting methods in our new AI courses. [Join now! (opens in a new tab)](https://dair-ai.thinkific.com/)

[Prompting Techniques](/techniques "Prompting Techniques")[Few-shot Prompting](/techniques/fewshot "Few-shot Prompting")

English

Light

* * *

Copyright © 2024 DAIR.AI


url: https://arxiv.org/html/2312.04076v1
title: Large Language Models are Good Prompt Learners for Low-Shot Image Classification
scrape_date: 2025-03-17T12:18:26.560255
content_type: markdown
hash: 9263cec5018e4724f3945158654cf0dceb089361308ae193b12d96075c0eaf64
content:
  1. 1 Introduction
  2. 2 Related Work
  3. 3 Approach
    1. 3.1 Preliminaries
    2. 3.2 Adaptive Prompt Learning with LLMs
    3. 3.3 The LLaMP Framework
    4. 3.4 Training and Inference
  4. 4 Experiments
    1. 4.1 Experiment Setup
    2. 4.2 Quantitative Evaluation
    3. 4.3 Ablation Study
  5. 5 Conclusion



HTML conversions [sometimes display errors](https://info.dev.arxiv.org/about/accessibility_html_error_messages.html) due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on.

  * failed: epic



Authors: achieve the best HTML results from your LaTeX submissions by selecting from this list of [supported packages](https://corpora.mathweb.org/corpus/arxmliv/tex_to_html/info/loaded_file).

License: arXiv.org perpetual non-exclusive license

arXiv:2312.04076v1 [cs.CV] 07 Dec 2023

# Large Language Models are Good Prompt Learners   
for Low-Shot Image Classification 

Zhaoheng Zheng Jingmin Wei Xuefeng Hu Haidong Zhu Ram Nevatia   
Viterbi School of Engineering   
University of Southern California   
{zhaoheng.zheng, jingminw, xuefengh, haidongz, nevatia}@usc.edu

###### Abstract

Low-shot image classification, where training images are limited or inaccessible, has benefited from recent progress on pre-trained vision-language (VL) models with strong generalizability, _e.g._ CLIP. Prompt learning methods built with VL models generate text features from the class names that only have confined class-specific information. Large Language Models (LLMs), with their vast encyclopedic knowledge, emerge as the complement. Thus, in this paper, we discuss the integration of LLMs to enhance pre-trained VL models, specifically on low-shot classification. However, the domain gap between language and vision blocks the direct application of LLMs. Thus, we propose LLaMP, Large Language Models as Prompt learners, that produces adaptive prompts for the CLIP text encoder, establishing it as the connecting bridge. Experiments show that, compared with other state-of-the-art prompt learning methods, LLaMP yields better performance on both zero-shot generalization and few-shot image classification, over a spectrum of 11 datasets.

##  1 Introduction

![Refer to caption](x1.png) (a) Visual Descriptions from an LLM.

![Refer to caption](x2.png) (b) LLMs’ Knowledge Boosts the Performance.

Figure 1: Demonstration of LLaMP: (a) LLMs can provide visual descriptions for fine-grained object categories; (b) Zero-shot base-to-novel generalization benefits from the LLM knowledge.

Low-shot image classification tasks, including few-shot and zero-shot variants, are to learn from a set of class names along with a limited or null set of images. Such capacities are crucial for the extension and generalization of vision systems. Vision-Language (VL) models trained on large-scale web data, such as CLIP [32] and ALIGN [14], provide a new paradigm due to their generalization capabilities that includes zero-shot classification, and have been used in recent work [18, 48, 49, 19, 17, 23, 24]. Due to the scarcity of images for training, methods built for both tasks rely heavily on merely category names as the source of class-specific knowledge, resulting in a shortage of distinguishable descriptions. Meanwhile, Large Language Models (LLMs), _e.g._ GPT-4 [28] and LLaMA [38, 39], have demonstrated their encyclopedic knowledge and thus can provide linguistic visual descriptions for objects. Here, we investigate how to leverage LLMs for low-shot image classification. 

The emergence of prompt learning has provided an efficient way to adapt large pre-trained models. Previous work has explored various strategies to prompt vision-language (VL) models, including vision-conditioned text prompt learning [48], joint VL prompt learning [18] and self-regulated VL prompts[19]. On the text side, regardless of the learning strategy, learned prompt vectors are shared across all categories. The only difference among text inputs is the class name. In low-shot scenarios where visual data is limited, the extraction of class-specific knowledge from textual inputs becomes essential. However, the current paradigm, which relies on the CLIP text encoder to distinguish between class names, faces challenges, particularly with fine-grained target categories. For example, in FGVCAircraft [25], the class name “Yak-40”, can barely provide any information for recognizing the object.

Large Language Models, trained with large text corpora, are good candidates to serve as the complement. As in Fig. 0(a), being queried about “Yak-40”, the LLM generates a sentence detailing the visual appearance of the Yak-40 that can be further parsed into noun phrases and integrated into text prompts, providing richer information, compared with the ordinary prompt. We also show in Fig. 0(b) that by simply incorporating noun phrases extracted from a LLM’s response, the performance of the ordinary CLIP models is improved by more than 1% without any training. Although recent prompt-learning based methods have shown notable improvements, it is non-trivial to apply them on textual visual descriptions generated by LLMs. Thus, instead of directly taking LLM generations as the textual input, we aim at producing class-specific representations by adapting LLMs to low-shot image classification.

One challenge of the adaption is the domain gap between vision and language. When trained exclusively with textual corpora, the latent feature space of a LLM significantly diverges from that of its visual counterpart. Even worse, the data scarcity under the low-shot scenario make it virtually impossible to align two spaces through plain contrastive loss. We argue that, the CLIP text encoder, which is trained to project features from the language domain into the joint VL domain, can serve as the bridge. Thus, we propose the LLaMP framework, Large Language Models as Prompt learners, which leverages LLMs to learn informative prompts for CLIP models. In LLaMP, we treat LLMs as the prompt learner of the CLIP text encoder. More specifically, for each object category, LLaMP extracts corresponding knowledge from the LLM and yields class-specific prompt vectors, which are further combined with class-agnostic prompt embeddings (as in previous approaches), and encoded by the CLIP text encoder. We design an efficient tuning pipeline to avoid fully fine-tuning the language model while performing effective adaptation.

Following the protocol in [48, 49], we evaluate LLaMP with two typical scenarios: zero-shot base-to-novel generalization [49] and few-shot image classification. For each scenario, we run LLaMP with 11 datasets covering a spectrum of tasks. On average, LLaMP achieves a 1.3% boost on the harmonic mean against the state-of-the-art PSRC [19], and 9.6% over the ordinary CLIP [32], on base-to-novel generalization. We also observe an average improvement of 0.94% on 16-shot image classification.

In summary, our approach makes use of Large Language Models to improve performance in low-shot image classification scenarios. The main contributions are: i) To the best of our knowledge, we are the first to investigate how to use the encyclopedic knowledge inherent in Large Language Models (LLMs) to enhance low-shot image classification; ii) We design a framework, LLaMP, to effectively adapt LLMs for image classification, without training the entire language model, and achieve state-of-the-art in both few-shot and zero-shot settings; iii) We conduct extensive analysis investigating the effectiveness of each components of LLaMP, and discuss the optimal setup for LLM-aided image classification.

##  2 Related Work

Large Language Models (LLMs). Recent years have witnessed remarkable progress in scaling up the size and capabilities of LLMs. Zhang _et al._ [47] first introduced a suite of transformers pre-trained at scale, followed by PaLM [6]. ChatGPT/GPT-4 [27, 28] emerged as a milestone conversational model, demonstrated impressive conversational abilities as a generalist model. Vicuna [5] further advanced by learning from ChatGPT, while LLaMA [38] demonstrates that larger scale training yields stronger foundation models. The subsequent LLaMA-2 [39] and PaLM-2 [2] achieved further gains in scale, efficiency and reasoning. Most recently, Almazrouei _et al._ [1] released Falcon, a 400B model.

Zero-Shot Learning (ZSL). ZSL stands in contrast to traditional fully-supervised paradigms. Instead of relying on direct visual training samples, it leverages side information that can be drawn from a multitude of non-visual domains, including attributes [22], word embeddings [40, 36], and descriptive texts [34]. Zhang _et al._ [46] designed an embedding model to bridge the gap between seen and unseen categories. Concurrently, studies like [4, 44, 50] have spotlighted that generative models can produce features for unseen categories. Moreover, Graph Convolution Networks (GCN) [20] has been explored in research such as [40, 16] to for further generalization.

Prompt Learning. With the progress in large-scale vision-language models, such as CLIP [32] and ALIGN [14], which reveal their capacity in zero-shot transferability, prompt learning has emerged as an efficient learning scheme, where learnable prompts are appended to the input to fine-tune models. For low-shot image classification, CoOp [49] and CoCoOp [48], which modeled context words as learnable vectors to automate prompt engineering, have shown significant improvements over regular CLIP. MaPLe [18] further employed a hierarchical multi-modal prompting strategy across transformer blocks for progressive feature modeling. Kan _et al._ [17] incorporated external knowledge by designing knowledge-aware prompts and adaptation head for better generalization. Lee _et al._ [23] used masked attention to prevent internal representation shift for better generalization. Khattak _et al._ [19] further improved prompt learning by guiding prompts to balance task-specific and task-agnostic knowledge via mutual agreement maximization and prompt ensemble.

##  3 Approach

![Refer to caption](x3.png) Figure 2: An Overview of the LLaMP Framework: We first generate the knowledge cache by passing the query prompt through the LLM 𝒟𝒟\mathcal{D}caligraphic_D and use the knowledge cache to encode 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT, resulting the adaptive prompts 𝒉~li=W⁢𝒉li+bisubscriptsuperscriptbold-~𝒉𝑖𝑙𝑊subscriptsuperscript𝒉𝑖𝑙superscript𝑏𝑖\bm{\tilde{h}}^{i}_{l}=W\bm{h}^{i}_{l}+b^{i}overbold_~ start_ARG bold_italic_h end_ARG start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT = italic_W bold_italic_h start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT + italic_b start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT for the CLIP text encoder. 𝒉~lsubscriptbold-~𝒉𝑙\bm{\tilde{h}}_{l}overbold_~ start_ARG bold_italic_h end_ARG start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT is combined with regular learnable prompts of 𝒢𝒢\mathcal{G}caligraphic_G to generate the final text feature vector 𝒈𝒑subscript𝒈𝒑\bm{g_{p}}bold_italic_g start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT. The image feature vector 𝒇𝒑subscript𝒇𝒑\bm{f_{p}}bold_italic_f start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT is obtained through a hybrid-tuning strategy combining prompt learning and low-rank adaptation (LoRA).

###  3.1 Preliminaries

Similar to previous CLIP-based learning approaches, we consider the classification problem as an image-text matching problem. We denote the image encoder and the text encoder, in CLIP-like models, as ℱℱ\mathcal{F}caligraphic_F and 𝒢𝒢\mathcal{G}caligraphic_G, parameterized by θℱsubscript𝜃ℱ\theta_{\mathcal{F}}italic_θ start_POSTSUBSCRIPT caligraphic_F end_POSTSUBSCRIPT and θ𝒢subscript𝜃𝒢\theta_{\mathcal{G}}italic_θ start_POSTSUBSCRIPT caligraphic_G end_POSTSUBSCRIPT, respectively. An input image 𝒙∈ℝC×H×W𝒙superscriptℝ𝐶𝐻𝑊\bm{x}\in\mathbb{R}^{C\times H\times W}bold_italic_x ∈ blackboard_R start_POSTSUPERSCRIPT italic_C × italic_H × italic_W end_POSTSUPERSCRIPT is split into M𝑀Mitalic_M equal-sized patches which are converted into a sequence of embeddings 𝒙~={𝒆c⁢l⁢s,𝒆1,𝒆2,…,𝒆M\bm{\tilde{x}}=\\{\bm{e}_{cls},\bm{e}_{1},\bm{e}_{2},\dots,\bm{e}_{M}overbold_~ start_ARG bold_italic_x end_ARG = { bold_italic_e start_POSTSUBSCRIPT italic_c italic_l italic_s end_POSTSUBSCRIPT , bold_italic_e start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , bold_italic_e start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT , … , bold_italic_e start_POSTSUBSCRIPT italic_M end_POSTSUBSCRIPT}. The visual input sequence 𝒙~bold-~𝒙\bm{\tilde{x}}overbold_~ start_ARG bold_italic_x end_ARG is encoded by the image encoder, producing the image feature 𝒇=ℱ⁢(𝒙~)𝒇ℱbold-~𝒙\bm{f}=\mathcal{F}(\bm{\tilde{x}})bold_italic_f = caligraphic_F ( overbold_~ start_ARG bold_italic_x end_ARG ). On the text side, the text label y𝑦yitalic_y and the associated name is formatted as “A photo of [STH]” and tokenized into a sequence of tokens 𝒚~={𝒕b⁢o⁢s,𝒕1,𝒕2,…,𝒕L,𝒕e⁢o⁢s}bold-~𝒚subscript𝒕𝑏𝑜𝑠subscript𝒕1subscript𝒕2…subscript𝒕𝐿subscript𝒕𝑒𝑜𝑠\bm{\tilde{y}}=\\{\bm{t}_{bos},\bm{t}_{1},\bm{t}_{2},\dots,\bm{t}_{L},\bm{t}_{% eos}\\}overbold_~ start_ARG bold_italic_y end_ARG = { bold_italic_t start_POSTSUBSCRIPT italic_b italic_o italic_s end_POSTSUBSCRIPT , bold_italic_t start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , bold_italic_t start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT , … , bold_italic_t start_POSTSUBSCRIPT italic_L end_POSTSUBSCRIPT , bold_italic_t start_POSTSUBSCRIPT italic_e italic_o italic_s end_POSTSUBSCRIPT }, where L𝐿Litalic_L is the length of input tokens. The input sequence is then encoded into 𝒈=𝒢⁢(𝒚~)𝒈𝒢bold-~𝒚\bm{g}=\mathcal{G}(\bm{\tilde{y}})bold_italic_g = caligraphic_G ( overbold_~ start_ARG bold_italic_y end_ARG ). For image classification, target class labels {1,2,…,C}12…𝐶\\{1,2,\dots,C\\}{ 1 , 2 , … , italic_C } are encoded into text features gisubscript𝑔𝑖g_{i}italic_g start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT. The classification is done by picking the class that has the highest similarity with the vision feature: y^=argmaxi𝒞⁢(𝒇,𝒈i)^𝑦subscriptargmax𝑖𝒞𝒇subscript𝒈𝑖\hat{y}=\operatorname*{argmax}_{i}\,\mathcal{C}(\bm{f},\bm{g}_{i})over^ start_ARG italic_y end_ARG = roman_argmax start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT caligraphic_C ( bold_italic_f , bold_italic_g start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ), where 𝒞𝒞\mathcal{C}caligraphic_C is the softmax cosine-similarity function 𝒞⁢(𝒇,𝒈)=exp⁡(𝒇⋅𝒈/τ)∑j=1Cexp⁡(𝒇⋅𝒈j/τ)𝒞𝒇𝒈⋅𝒇𝒈𝜏superscriptsubscript𝑗1𝐶⋅𝒇subscript𝒈𝑗𝜏\mathcal{C}(\bm{f},\bm{g})=\frac{\exp{(\bm{f}\cdot\bm{g}/\tau)}}{\sum_{j=1}^{C% }\exp{(\bm{f}\cdot\bm{g}_{j}/\tau)}}caligraphic_C ( bold_italic_f , bold_italic_g ) = divide start_ARG roman_exp ( bold_italic_f ⋅ bold_italic_g / italic_τ ) end_ARG start_ARG ∑ start_POSTSUBSCRIPT italic_j = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_C end_POSTSUPERSCRIPT roman_exp ( bold_italic_f ⋅ bold_italic_g start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT / italic_τ ) end_ARG with temperature τ𝜏\tauitalic_τ.

Multimodal Prompting Learning.  Given the size of the CLIP model, fine-tuning the entire model becomes infeasible. As both image and text encoders are built with standard transformer architecture, prompt learning, which tunes the model by combining trainable prompts with hidden states has been applied on the text encoder [48, 49], the image encoder [15, 41, 42], or both [18, 19, 33]. Similar to [19, 33], we build our method following the vision-language prompting paradigm, with deep prompting [15, 19], which not only insert prompts to the input layer, but to later encoder layers.

More specifically, for each transformer layer that takes prompts, we define V𝑉Vitalic_V learnable visual prompts 𝒑v={pv1,pv2,…,pvV}subscript𝒑𝑣superscriptsubscript𝑝𝑣1superscriptsubscript𝑝𝑣2…superscriptsubscript𝑝𝑣𝑉\bm{p}_{v}=\\{p_{v}^{1},p_{v}^{2},\dots,p_{v}^{V}\\}bold_italic_p start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT = { italic_p start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 end_POSTSUPERSCRIPT , italic_p start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT , … , italic_p start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_V end_POSTSUPERSCRIPT } and T𝑇Titalic_T learnable language prompts 𝒑t={pt1,pt2,…,ptT}subscript𝒑𝑡superscriptsubscript𝑝𝑡1superscriptsubscript𝑝𝑡2…superscriptsubscript𝑝𝑡𝑇\bm{p}_{t}=\\{p_{t}^{1},p_{t}^{2},\dots,p_{t}^{T}\\}bold_italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT = { italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 end_POSTSUPERSCRIPT , italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT , … , italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_T end_POSTSUPERSCRIPT }. For the i𝑖iitalic_i-th vision encoder layer, visual prompts 𝒑visuperscriptsubscript𝒑𝑣𝑖\bm{p}_{v}^{i}bold_italic_p start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT are appended to input embeddings: 𝒙~𝒑i={𝒆c⁢l⁢si,𝒆1i,𝒆2i,…,𝒆Mi,𝒑vi}superscriptsubscriptbold-~𝒙𝒑𝑖superscriptsubscript𝒆𝑐𝑙𝑠𝑖superscriptsubscript𝒆1𝑖superscriptsubscript𝒆2𝑖…superscriptsubscript𝒆𝑀𝑖superscriptsubscript𝒑𝑣𝑖\bm{\tilde{x}_{p}}^{i}=\\{\bm{e}_{cls}^{i},\bm{e}_{1}^{i},\bm{e}_{2}^{i},\dots,% \bm{e}_{M}^{i},\bm{p}_{v}^{i}\\}overbold_~ start_ARG bold_italic_x end_ARG start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT = { bold_italic_e start_POSTSUBSCRIPT italic_c italic_l italic_s end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , bold_italic_e start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , bold_italic_e start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , … , bold_italic_e start_POSTSUBSCRIPT italic_M end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , bold_italic_p start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT }. The prompt-augmented vision feature, 𝒇𝒑=ℱ⁢(𝒙~𝒑)subscript𝒇𝒑ℱsubscriptbold-~𝒙𝒑\bm{f_{p}}=\mathcal{F}(\bm{\tilde{x}_{p}})bold_italic_f start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT = caligraphic_F ( overbold_~ start_ARG bold_italic_x end_ARG start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT ), is produced by jointly encoding prompts and the image. As the ViT [9] architecture in CLIP adopts the bi-directional attention mechanism, the placement of 𝒑vsubscript𝒑𝑣\bm{p}_{v}bold_italic_p start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT has no effect on 𝒇𝒑subscript𝒇𝒑\bm{f_{p}}bold_italic_f start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT. On the language side, prompts are concatenated with the input of the i𝑖iitalic_i-th text encoder: 𝒚~𝒑i={𝒕b⁢o⁢si,𝒑ti,𝒕1i,𝒕2i,…,𝒕Li,𝒕e⁢o⁢si}superscriptsubscriptbold-~𝒚𝒑𝑖superscriptsubscript𝒕𝑏𝑜𝑠𝑖subscriptsuperscript𝒑𝑖𝑡superscriptsubscript𝒕1𝑖superscriptsubscript𝒕2𝑖…superscriptsubscript𝒕𝐿𝑖superscriptsubscript𝒕𝑒𝑜𝑠𝑖\bm{\tilde{y}_{p}}^{i}=\\{\bm{t}_{bos}^{i},\bm{p}^{i}_{t},\bm{t}_{1}^{i},\bm{t}% _{2}^{i},\dots,\bm{t}_{L}^{i},\bm{t}_{eos}^{i}\\}overbold_~ start_ARG bold_italic_y end_ARG start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT = { bold_italic_t start_POSTSUBSCRIPT italic_b italic_o italic_s end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , bold_italic_p start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT , bold_italic_t start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , bold_italic_t start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , … , bold_italic_t start_POSTSUBSCRIPT italic_L end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , bold_italic_t start_POSTSUBSCRIPT italic_e italic_o italic_s end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT }. 𝒚𝒑~bold-~subscript𝒚𝒑\bm{\tilde{y_{p}}}overbold_~ start_ARG bold_italic_y start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT end_ARG is further processed by the text encoder, resulting in the prompt-augmented language feature 𝒈𝒑=𝒢⁢(𝒚~𝒑)subscript𝒈𝒑𝒢subscriptbold-~𝒚𝒑\bm{g_{p}}=\mathcal{G}(\bm{\tilde{y}_{p}})bold_italic_g start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT = caligraphic_G ( overbold_~ start_ARG bold_italic_y end_ARG start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT ). More specifically, prompts to the first layer 𝒑t1superscriptsubscript𝒑𝑡1\bm{p}_{t}^{1}bold_italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT start_POSTSUPERSCRIPT 1 end_POSTSUPERSCRIPT are initialized with the embeddings of “A photo of a”. 

Low-Rank Adaptation [13] (LoRA). As a parameter-efficient tuning technique, LoRA is designed to adapt large transformer model without updating original model weights. The LoRA technique is, in particular, applied to linear projection layers. More specifically, for a linear layer with weight W0∈ℝd×ksubscript𝑊0superscriptℝ𝑑𝑘W_{0}\in\mathbb{R}^{d\times k}italic_W start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ∈ blackboard_R start_POSTSUPERSCRIPT italic_d × italic_k end_POSTSUPERSCRIPT, LoRA creates Δ⁢WΔ𝑊\Delta Wroman_Δ italic_W by learning two low rank matrices B∈ℝd×r𝐵superscriptℝ𝑑𝑟B\in\mathbb{R}^{d\times r}italic_B ∈ blackboard_R start_POSTSUPERSCRIPT italic_d × italic_r end_POSTSUPERSCRIPT and A∈ℝr×k𝐴superscriptℝ𝑟𝑘A\in\mathbb{R}^{r\times k}italic_A ∈ blackboard_R start_POSTSUPERSCRIPT italic_r × italic_k end_POSTSUPERSCRIPT:

| 𝒉=(W0+Δ⁢W)⁢𝒙=W0⁢𝒙+B⁢A⁢𝒙.𝒉subscript𝑊0Δ𝑊𝒙subscript𝑊0𝒙𝐵𝐴𝒙\bm{h}=(W_{0}+\Delta W)\bm{x}=W_{0}\bm{x}+BA\bm{x}.bold_italic_h = ( italic_W start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT + roman_Δ italic_W ) bold_italic_x = italic_W start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT bold_italic_x + italic_B italic_A bold_italic_x . |  | (1)  
---|---|---|---  
  
We adopt a hybrid tuning scheme on the vision encoder, which performs prompt learning on the first few layers and applies LoRA on the rest.

###  3.2 Adaptive Prompt Learning with LLMs

The goal of prompt tuning is to find an set of optimal prompts 𝒑={𝒑v,𝒑t}𝒑subscript𝒑𝑣subscript𝒑𝑡\bm{p}=\\{\bm{p}_{v},\bm{p}_{t}\\}bold_italic_p = { bold_italic_p start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT , bold_italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT } which maximizes the log likelihood of P⁢(x,y|θℱ,θ𝒢)𝑃𝑥conditional𝑦subscript𝜃ℱsubscript𝜃𝒢P(x,y|\theta_{\mathcal{F}},\theta_{\mathcal{G}})italic_P ( italic_x , italic_y | italic_θ start_POSTSUBSCRIPT caligraphic_F end_POSTSUBSCRIPT , italic_θ start_POSTSUBSCRIPT caligraphic_G end_POSTSUBSCRIPT ) over target downstream distribution (𝒙,𝒚)∼(𝑿,𝒀)similar-to𝒙𝒚𝑿𝒀(\bm{x},\bm{y})\sim(\bm{X},\bm{Y})( bold_italic_x , bold_italic_y ) ∼ ( bold_italic_X , bold_italic_Y ):

| 𝒑=argmax𝒑𝔼(𝒙,𝒚)∼(𝑿,𝒀)⁢log⁡𝒞⁢(ℱ⁢(𝒙;𝒑v),𝒢⁢(𝒚;𝒑t))𝒑subscriptargmax𝒑subscript𝔼similar-to𝒙𝒚𝑿𝒀𝒞ℱ𝒙subscript𝒑𝑣𝒢𝒚subscript𝒑𝑡\bm{p}=\operatorname*{argmax}_{\bm{p}}\mathbb{E}_{(\bm{x},\bm{y})\sim(\bm{X},% \bm{Y})}\log\mathcal{C}(\mathcal{F}(\bm{x};\bm{p}_{v}),\mathcal{G}(\bm{y};\bm{% p}_{t}))bold_italic_p = roman_argmax start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT blackboard_E start_POSTSUBSCRIPT ( bold_italic_x , bold_italic_y ) ∼ ( bold_italic_X , bold_italic_Y ) end_POSTSUBSCRIPT roman_log caligraphic_C ( caligraphic_F ( bold_italic_x ; bold_italic_p start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT ) , caligraphic_G ( bold_italic_y ; bold_italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT ) ) |  | (2)  
---|---|---|---  
  
However, the 𝒑𝒑\bm{p}bold_italic_p optimized through Eqn. 2 has two issues. First, 𝒑𝒑\bm{p}bold_italic_p is shared for all categories for the downstream task, while the optimal prompt for each category might be different. Second, in low-shot scenarios, 𝒑𝒑\bm{p}bold_italic_p are usually empirically estimated from a limited training-set 𝑿t⁢r⁢a⁢i⁢nsuperscript𝑿𝑡𝑟𝑎𝑖𝑛\bm{X}^{train}bold_italic_X start_POSTSUPERSCRIPT italic_t italic_r italic_a italic_i italic_n end_POSTSUPERSCRIPT with limited categories {1,2,…,Cb⁢a⁢s⁢e}12…superscript𝐶𝑏𝑎𝑠𝑒\\{1,2,...,C^{base}\\}{ 1 , 2 , … , italic_C start_POSTSUPERSCRIPT italic_b italic_a italic_s italic_e end_POSTSUPERSCRIPT }, and therefore such 𝒑𝒑\bm{p}bold_italic_p can often be over-fitted to the small training-set 𝑿t⁢r⁢a⁢i⁢nsuperscript𝑿𝑡𝑟𝑎𝑖𝑛\bm{X}^{train}bold_italic_X start_POSTSUPERSCRIPT italic_t italic_r italic_a italic_i italic_n end_POSTSUPERSCRIPT and fail to generalize to novel categories outside {1,2,…,Cb⁢a⁢s⁢e}12…superscript𝐶𝑏𝑎𝑠𝑒\\{1,2,...,C^{base}\\}{ 1 , 2 , … , italic_C start_POSTSUPERSCRIPT italic_b italic_a italic_s italic_e end_POSTSUPERSCRIPT }.

To overcome these problems, we propose to learn a meta function on the language side 𝒑t=Θ⁢(y)subscript𝒑𝑡Θ𝑦\bm{p}_{t}=\Theta(y)bold_italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT = roman_Θ ( italic_y ) which can adaptively estimate the optimal prompt for each category. An intuitive way to estimate proper prompts 𝒑𝒑\bm{p}bold_italic_p for category name y𝑦yitalic_y is to take advantage of the knowledge of the pre-trained Large Lanuage Models (LLM) 𝒟𝒟\mathcal{D}caligraphic_D and extract discriminative descriptions of category y𝑦yitalic_y. For example, given the input text 𝒛𝒛\bm{z}bold_italic_z:“Describe {y}”,

| 𝒑t={p1,p2,…,pk}=𝒟⁢(𝒛).subscript𝒑𝑡subscript𝑝1subscript𝑝2…subscript𝑝𝑘𝒟𝒛\bm{p}_{t}=\\{p_{1},p_{2},...,p_{k}\\}=\mathcal{D}(\bm{z}).bold_italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT = { italic_p start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , italic_p start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT , … , italic_p start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT } = caligraphic_D ( bold_italic_z ) . |  | (3)  
---|---|---|---  
  
while pisubscript𝑝𝑖p_{i}italic_p start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT being sequencially generated by 𝒟𝒟\mathcal{D}caligraphic_D such that

| pisubscript𝑝𝑖\displaystyle p_{i}italic_p start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | =𝒟⁢(𝒛,t1,…,ti−1)=𝒟(i)⁢(𝒛)absent𝒟𝒛subscript𝑡1…subscript𝑡𝑖1superscript𝒟𝑖𝒛\displaystyle=\mathcal{D}(\bm{z},t_{1},...,t_{i-1})=\mathcal{D}^{(i)}(\bm{z})= caligraphic_D ( bold_italic_z , italic_t start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , … , italic_t start_POSTSUBSCRIPT italic_i - 1 end_POSTSUBSCRIPT ) = caligraphic_D start_POSTSUPERSCRIPT ( italic_i ) end_POSTSUPERSCRIPT ( bold_italic_z ) |  | (4)  
---|---|---|---|---  
| tisubscript𝑡𝑖\displaystyle t_{i}italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT | =ℳ⁢(pi),absentℳsubscript𝑝𝑖\displaystyle=\mathcal{M}(p_{i}),= caligraphic_M ( italic_p start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) , |   
  
where 𝒟(i)superscript𝒟𝑖\mathcal{D}^{(i)}caligraphic_D start_POSTSUPERSCRIPT ( italic_i ) end_POSTSUPERSCRIPT is the i𝑖iitalic_i-th forward iteration of 𝒟𝒟\mathcal{D}caligraphic_D, and ℳℳ\mathcal{M}caligraphic_M maps continues hidden states into discrete language tokens. To accerelate the process and to obtain p𝑝pitalic_p in one pass, we approximate the above process with K𝐾Kitalic_K learnable prompts 𝒑l={θ1,…,θK}subscript𝒑𝑙subscript𝜃1…subscript𝜃𝐾\bm{p}_{l}=\\{\theta_{1},...,\theta_{K}\\}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT = { italic_θ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , … , italic_θ start_POSTSUBSCRIPT italic_K end_POSTSUBSCRIPT } so that

| 𝒑tsubscript𝒑𝑡\displaystyle\bm{p}_{t}bold_italic_p start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT | =Θ⁢(y)=𝒟⁢({θ1,…,θK}|𝒛)absentΘ𝑦𝒟conditionalsubscript𝜃1…subscript𝜃𝐾𝒛\displaystyle=\Theta(y)=\mathcal{D}(\\{\theta_{1},...,\theta_{K}\\}|\bm{z})= roman_Θ ( italic_y ) = caligraphic_D ( { italic_θ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , … , italic_θ start_POSTSUBSCRIPT italic_K end_POSTSUBSCRIPT } | bold_italic_z ) |  | (5)  
---|---|---|---|---  
  
Discussion. While Large Language Models (LLMs) possess robust foundational knowledge within the linguistic domain, it is not feasible to directly substitute the text encoder of CLIP with an LLM. The reason lies in the inherent divergence between the LLM’s latent space, which is purely language-oriented, and the image-focused latent space of vision encoders. Attempting a direct alignment via contrastive learning would require an extensive dataset that is typically beyond the scope of low-shot learning. To bridge this gap, we introduce LLaMP—an adaptive prompt learning framework that leverages the LLM to craft class-specific prompt vectors, to reinforce the text encoder for low-shot image classification.

###  3.3 The LLaMP Framework

Fig. 2 shows an overview of the LLaMP framework. For convenience, we denote the decoder-only LLM as 𝒟𝒟\mathcal{D}caligraphic_D. The input to the decoder 𝒟𝒟\mathcal{D}caligraphic_D consists of two components: textual prompts 𝒚𝒚\bm{y}bold_italic_y in the form of sentences, tokenized as 𝒚~bold-~𝒚\bm{\tilde{y}}overbold_~ start_ARG bold_italic_y end_ARG, and learnable prompts 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT. We append prompt embeddings to the end of the input sequence and obtain the last hidden states of 𝒟𝒟\mathcal{D}caligraphic_D as the feature 𝒉lsubscript𝒉𝑙\bm{h}_{l}bold_italic_h start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT:

| 𝒉l=𝒟(𝒚~,𝒑l)[L+1:L+K],L=Length(𝒚~).\bm{h}_{l}=\mathcal{D}(\bm{\tilde{y}},\bm{p}_{l})[L+1:L+K],L=\text{Length}(\bm% {\tilde{y}}).bold_italic_h start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT = caligraphic_D ( overbold_~ start_ARG bold_italic_y end_ARG , bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT ) [ italic_L + 1 : italic_L + italic_K ] , italic_L = Length ( overbold_~ start_ARG bold_italic_y end_ARG ) . |  | (6)  
---|---|---|---  
  
Hidden states of 𝒟𝒟\mathcal{D}caligraphic_D are then mapped to the input space of the CLIP text encoder by the projection matrix W∈ℝd1×d2𝑊superscriptℝsubscript𝑑1subscript𝑑2W\in\mathbb{R}^{d_{1}\times d_{2}}italic_W ∈ blackboard_R start_POSTSUPERSCRIPT italic_d start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT × italic_d start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT, where d1subscript𝑑1d_{1}italic_d start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT and d2subscript𝑑2d_{2}italic_d start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT are respectively the hidden sizes of the LLM 𝒟𝒟\mathcal{D}caligraphic_D and the CLIP text encoder 𝒢𝒢\mathcal{G}caligraphic_G. A set of prompt-specific biases b∈ℝK×d2𝑏superscriptℝ𝐾subscript𝑑2b\in\mathbb{R}^{K\times d_{2}}italic_b ∈ blackboard_R start_POSTSUPERSCRIPT italic_K × italic_d start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT are added:

| 𝒉~l=W⁢𝒉l+bsubscriptbold-~𝒉𝑙𝑊subscript𝒉𝑙𝑏\bm{\tilde{h}}_{l}=W\bm{h}_{l}+boverbold_~ start_ARG bold_italic_h end_ARG start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT = italic_W bold_italic_h start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT + italic_b |  | (7)  
---|---|---|---  
  
We combine 𝒉~𝒍subscriptbold-~𝒉𝒍\bm{\tilde{h}_{l}}overbold_~ start_ARG bold_italic_h end_ARG start_POSTSUBSCRIPT bold_italic_l end_POSTSUBSCRIPT from LLM with regular learnable prompts, as in previous approaches [19], to construct the input for CLIP text encoder. Similar to deep prompting [15, 19], we create layer-specific prompts through different W𝑊Witalic_W matrices and b𝑏bitalic_b vectors. For the i𝑖iitalic_i-th layer, we let 𝒉~li=Wi⁢𝒉l+bisuperscriptsubscriptbold-~𝒉𝑙𝑖superscript𝑊𝑖subscript𝒉𝑙superscript𝑏𝑖\bm{\tilde{h}}_{l}^{i}=W^{i}\bm{h}_{l}+b^{i}overbold_~ start_ARG bold_italic_h end_ARG start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT = italic_W start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT bold_italic_h start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT + italic_b start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT and the entire sequence is constructed as

| 𝒚~𝒍i={𝒕b⁢o⁢si,𝒑ti,𝒕1i,𝒕2i,…,𝒕Li,𝒕e⁢o⁢si,𝒉~𝒍𝒊}superscriptsubscriptbold-~𝒚𝒍𝑖superscriptsubscript𝒕𝑏𝑜𝑠𝑖subscriptsuperscript𝒑𝑖𝑡superscriptsubscript𝒕1𝑖superscriptsubscript𝒕2𝑖…superscriptsubscript𝒕𝐿𝑖superscriptsubscript𝒕𝑒𝑜𝑠𝑖superscriptsubscriptbold-~𝒉𝒍𝒊\bm{\tilde{y}_{l}}^{i}=\\{\bm{t}_{bos}^{i},\bm{p}^{i}_{t},\bm{t}_{1}^{i},\bm{t}% _{2}^{i},\dots,\bm{t}_{L}^{i},\bm{t}_{eos}^{i},\bm{\tilde{h}_{l}^{i}}\\}overbold_~ start_ARG bold_italic_y end_ARG start_POSTSUBSCRIPT bold_italic_l end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT = { bold_italic_t start_POSTSUBSCRIPT italic_b italic_o italic_s end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , bold_italic_p start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT , bold_italic_t start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , bold_italic_t start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , … , bold_italic_t start_POSTSUBSCRIPT italic_L end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , bold_italic_t start_POSTSUBSCRIPT italic_e italic_o italic_s end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT , overbold_~ start_ARG bold_italic_h end_ARG start_POSTSUBSCRIPT bold_italic_l end_POSTSUBSCRIPT start_POSTSUPERSCRIPT bold_italic_i end_POSTSUPERSCRIPT } |  | (8)  
---|---|---|---  
  
LLM Knowledge Cache. A Large Language Model (LLM), as implied by its name, typically comprises billions of parameters. For example, the most compact LLaMA [38, 39] model has 7B parameters. Thus, even performing prompt learning on a LLM become impractical. The memory consumption to store gradients for back propagation can go beyond the limit of mainstream GPUs. Instead, the causal attention mechanism inherent in decoder-only LLMs, where the embedding of an input token only depends on the preceding tokens, facilitates a feasible workaround.

As previously mentioned, the prompt embeddings 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT are appended to the end of text tokens 𝒚~bold-~𝒚\bm{\tilde{y}}overbold_~ start_ARG bold_italic_y end_ARG. According to the causal attention mechanism, 𝒚~bold-~𝒚\bm{\tilde{y}}overbold_~ start_ARG bold_italic_y end_ARG is encoded independently of 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT. Thus, we design a two-stage process, where we create the LLM knowledge cache by passing 𝒚~bold-~𝒚\bm{\tilde{y}}overbold_~ start_ARG bold_italic_y end_ARG through 𝒟𝒟\mathcal{D}caligraphic_D and leverage the cache to convert 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT into class-specific embeddings for the CLIP text encoder 𝒢𝒢\mathcal{G}caligraphic_G.

To compute the attention of a token, the only dependency is the Key and Value vectors from the preceding tokens. Thus, we adopt the KV-cache [31, 43], a technique used in inference acceleration of LLMs, to create the knowledge cache. At the first stage, we pass text tokens 𝒚~bold-~𝒚\bm{\tilde{y}}overbold_~ start_ARG bold_italic_y end_ARG through the language model 𝒟𝒟\mathcal{D}caligraphic_D and save the Keys and Values as the knowledge cache for the second stage. Once computed, the knowledge cache remains fixed throughout the entire training process and bears the information that is needed for further computation. Thus, in LLaMP, we leverage the knowledge cache obtained at the first stage to generate class-specific prompt embeddings.

At the second stage, we create class-specific prompt embeddings from the pre-computed knowledge cache. As 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT is not initialized in the natural language domain, it need not pass through the entire LLM; instead, we insert those prompts 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT to the last layer of the LLM 𝒟Nsubscript𝒟𝑁\mathcal{D}_{N}caligraphic_D start_POSTSUBSCRIPT italic_N end_POSTSUBSCRIPT. It is achieved by encoding them alongside the cache from 𝒚~bold-~𝒚\bm{\tilde{y}}overbold_~ start_ARG bold_italic_y end_ARG, as in

| 𝑯l=𝒟N⁢(𝑲𝒚~,𝑽𝒚~,𝒑l),subscript𝑯𝑙subscript𝒟𝑁subscript𝑲bold-~𝒚subscript𝑽bold-~𝒚subscript𝒑𝑙\bm{H}_{l}=\mathcal{D}_{N}(\bm{K}_{\bm{\tilde{y}}},\bm{V}_{\bm{\tilde{y}}},\bm% {p}_{l}),bold_italic_H start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT = caligraphic_D start_POSTSUBSCRIPT italic_N end_POSTSUBSCRIPT ( bold_italic_K start_POSTSUBSCRIPT overbold_~ start_ARG bold_italic_y end_ARG end_POSTSUBSCRIPT , bold_italic_V start_POSTSUBSCRIPT overbold_~ start_ARG bold_italic_y end_ARG end_POSTSUBSCRIPT , bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT ) , |  | (9)  
---|---|---|---  
  
where 𝑲𝒚~,𝑽𝒚~subscript𝑲bold-~𝒚subscript𝑽bold-~𝒚\bm{K}_{\bm{\tilde{y}}},\bm{V}_{\bm{\tilde{y}}}bold_italic_K start_POSTSUBSCRIPT overbold_~ start_ARG bold_italic_y end_ARG end_POSTSUBSCRIPT , bold_italic_V start_POSTSUBSCRIPT overbold_~ start_ARG bold_italic_y end_ARG end_POSTSUBSCRIPT represent the knowledge cache. This design enables LLaMP to efficiently learn informative prompt embeddings for the CLIP encoder 𝒢𝒢\mathcal{G}caligraphic_G. It accomplishes this by incurring modest training costs, compared with training the entire LLM. Simultaneously, it maintains the essential knowledge inherent in the LLM decoder 𝒟𝒟\mathcal{D}caligraphic_D.

Training Targets of LLaMP. Although the training strategy in Eqn. 9 has reduced the number of learnable parameters, a full decoder inside a LLM still consists of an enormous number of parameters. For example, one layer in LLaMA-7B bears 200M parameters, making training of the entire layer costly. As the goal is to leverage the knowledge from LLM, altering a full layer can lead to the loss of knowledge. Thus, as shown in Fig. 2, a typical decoder layer has two major components: the self attention module, consisting Query, Key, Value and Output projection layers, and the Feed-Forward Network (FFN). LLaMP targets the Query and Output projection layer inside the self-attention module. By updating the Query layer, LLM prompts 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT are learned to distill pertinent information from the knowledge cache and the Output layer projects it to the latent space. We keep the Key and Value layers frozen to ensure the alignment between 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT and knowledge cache. We leave the FFN unchanged to preserve the knowledge. Further discussions regarding these choices are made in Sec. 4.3.

Textual Priors from Pre-Generated Responses. We extend the initial prompt, “In one sentence, describe the distinctive appearance of [STH]”, by incorporating the response generated by the language model into the input sequence. This approach enriches the base content: the generated text provides a clear and explicit description of the object’s appearance, acting as a valuable informative prior for language model adaptation. However, it is common for responses from an LLM to include filler words like “sure” for sentence structure coherence. To refine the input, we parse the noun phrases from the LLM’s response through spaCy [12], an NLP engine, and merge them with the initial prompt, forming a more focused and informative language prior.

Textual Augmentations. Following the insights of Khattak et al. [19], which highlight the performance benefits of diverse textual inputs, we aim to further augment the text inputs used in the CLIP text encoder. Our approach, building upon the methods in [19, 48], incorporates hand-crafted templates and expands their diversity through a two-step process: i) We introduce noun phrases into the existing templates for CLIP, for example, transforming “A photo of [STH]” to “A photo of [STH] with [NP]”, thereby enriching the descriptive content; ii) We create a variety of new prompt templates for the LLM similar to “In one sentence, describe the distinctive appearance of [STH]” through GPT-4 [28], to further diversify the text input.

###  3.4 Training and Inference

Similar to PSRC [19], our objective function consists of three components: The main cross-entropy loss ℒC⁢Esubscriptℒ𝐶𝐸\mathcal{L}_{CE}caligraphic_L start_POSTSUBSCRIPT italic_C italic_E end_POSTSUBSCRIPT, feature-level L1 regularization ℒl⁢1subscriptℒ𝑙1\mathcal{L}_{l1}caligraphic_L start_POSTSUBSCRIPT italic_l 1 end_POSTSUBSCRIPT, and soft distillation loss ℒd⁢i⁢s⁢tsubscriptℒ𝑑𝑖𝑠𝑡\mathcal{L}_{dist}caligraphic_L start_POSTSUBSCRIPT italic_d italic_i italic_s italic_t end_POSTSUBSCRIPT. Given 𝒞𝒞\mathcal{C}caligraphic_C training categories and 𝒩𝒩\mathcal{N}caligraphic_N training samples, ℒC⁢Esubscriptℒ𝐶𝐸\mathcal{L}_{CE}caligraphic_L start_POSTSUBSCRIPT italic_C italic_E end_POSTSUBSCRIPT is defined as

| ℒC⁢E=−1𝒩⁢∑ilog⁡exp⁡(𝒇𝒑i⋅𝒈𝒑i/τ)∑exp⁡(𝒇𝒑i⋅𝒈𝒑j/τ).subscriptℒ𝐶𝐸1𝒩subscript𝑖⋅superscriptsubscript𝒇𝒑𝑖superscriptsubscript𝒈𝒑𝑖𝜏⋅superscriptsubscript𝒇𝒑𝑖superscriptsubscript𝒈𝒑𝑗𝜏\mathcal{L}_{CE}=-\frac{1}{\mathcal{N}}\sum_{i}\log\frac{\exp{(\bm{f_{p}}^{i}% \cdot\bm{g_{p}}^{i}/\tau)}}{\sum\exp{(\bm{f_{p}}^{i}\cdot\bm{g_{p}}^{j}/\tau)}}.caligraphic_L start_POSTSUBSCRIPT italic_C italic_E end_POSTSUBSCRIPT = - divide start_ARG 1 end_ARG start_ARG caligraphic_N end_ARG ∑ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT roman_log divide start_ARG roman_exp ( bold_italic_f start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT ⋅ bold_italic_g start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT / italic_τ ) end_ARG start_ARG ∑ roman_exp ( bold_italic_f start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT ⋅ bold_italic_g start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_j end_POSTSUPERSCRIPT / italic_τ ) end_ARG . |  | (10)  
---|---|---|---  
  
The L1 regularization is computed between learned features 𝒇𝒑,𝒈𝒑subscript𝒇𝒑subscript𝒈𝒑\bm{f_{p}},\bm{g_{p}}bold_italic_f start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT , bold_italic_g start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT and pre-trained CLIP features 𝒇^,𝒈^bold-^𝒇bold-^𝒈\bm{\hat{f}},\bm{\hat{g}}overbold_^ start_ARG bold_italic_f end_ARG , overbold_^ start_ARG bold_italic_g end_ARG:

| ℒl⁢1=1𝒩⁢∑iλv⁢|𝒇𝒑i−𝒇^i|+1C⁢∑iλt⁢|𝒈𝒑i−𝒈^i|,subscriptℒ𝑙11𝒩subscript𝑖subscript𝜆𝑣superscriptsubscript𝒇𝒑𝑖superscriptbold-^𝒇𝑖1𝐶subscript𝑖subscript𝜆𝑡superscriptsubscript𝒈𝒑𝑖superscriptbold-^𝒈𝑖\mathcal{L}_{l1}=\frac{1}{\mathcal{N}}\sum_{i}\lambda_{v}|\bm{f_{p}}^{i}-\bm{% \hat{f}}^{i}|+\frac{1}{C}\sum_{i}\lambda_{t}|\bm{g_{p}}^{i}-\bm{\hat{g}}^{i}|,caligraphic_L start_POSTSUBSCRIPT italic_l 1 end_POSTSUBSCRIPT = divide start_ARG 1 end_ARG start_ARG caligraphic_N end_ARG ∑ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT italic_λ start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT | bold_italic_f start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT - overbold_^ start_ARG bold_italic_f end_ARG start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT | + divide start_ARG 1 end_ARG start_ARG italic_C end_ARG ∑ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT italic_λ start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT | bold_italic_g start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT - overbold_^ start_ARG bold_italic_g end_ARG start_POSTSUPERSCRIPT italic_i end_POSTSUPERSCRIPT | , |  | (11)  
---|---|---|---  
  
where λvsubscript𝜆𝑣\lambda_{v}italic_λ start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT and λtsubscript𝜆𝑡\lambda_{t}italic_λ start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT are coefficients. The prediction of LLaMP is further bound by the KL-Divergence between predicted distributions of LLaMP and vanilla CLIP:

| ℒd⁢i⁢s⁢t=λd⁢i⁢s⁢t⁢DK⁢L⁢(𝒇𝒑⋅𝒈𝒑,𝒇^⋅𝒈^).subscriptℒ𝑑𝑖𝑠𝑡subscript𝜆𝑑𝑖𝑠𝑡subscript𝐷𝐾𝐿⋅subscript𝒇𝒑subscript𝒈𝒑⋅bold-^𝒇bold-^𝒈\mathcal{L}_{dist}=\lambda_{dist}D_{KL}(\bm{f_{p}}\cdot\bm{g_{p}},\bm{\hat{f}}% \cdot\bm{\hat{g}}).caligraphic_L start_POSTSUBSCRIPT italic_d italic_i italic_s italic_t end_POSTSUBSCRIPT = italic_λ start_POSTSUBSCRIPT italic_d italic_i italic_s italic_t end_POSTSUBSCRIPT italic_D start_POSTSUBSCRIPT italic_K italic_L end_POSTSUBSCRIPT ( bold_italic_f start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT ⋅ bold_italic_g start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT , overbold_^ start_ARG bold_italic_f end_ARG ⋅ overbold_^ start_ARG bold_italic_g end_ARG ) . |  | (12)  
---|---|---|---  
  
We sum all three losses up as the final objecttive function: ℒ=ℒC⁢E+ℒl⁢1+ℒd⁢i⁢s⁢t.ℒsubscriptℒ𝐶𝐸subscriptℒ𝑙1subscriptℒ𝑑𝑖𝑠𝑡\mathcal{L}=\mathcal{L}_{CE}+\mathcal{L}_{l1}+\mathcal{L}_{dist}.caligraphic_L = caligraphic_L start_POSTSUBSCRIPT italic_C italic_E end_POSTSUBSCRIPT + caligraphic_L start_POSTSUBSCRIPT italic_l 1 end_POSTSUBSCRIPT + caligraphic_L start_POSTSUBSCRIPT italic_d italic_i italic_s italic_t end_POSTSUBSCRIPT .

During training, we randomly sample one LLM template as the input of LLaMP for each batch. For inference, we compute the probability distribution predicted from each input template and average them.

##  4 Experiments

Method |  _Average_ |  ImageNet [8] |  Caltech101 [10] |  OxfordPets [29]  
---|---|---|---|---  
Base | Novel | HM | Base | Novel | HM | Base | Novel | HM | Base | Novel | HM  
CLIP [32] | 69.34 | 74.22 | 71.70 | 72.43 | 68.14 | 70.22 | 96.84 | 94.00 | 95.40 | 91.17 | 97.26 | 94.12  
CoOp [49] | 82.69 | 63.22 | 71.66 | 76.47 | 67.88 | 71.92 | 98.00 | 89.81 | 93.73 | 93.67 | 95.29 | 94.47  
CoCoOp [48] | 80.47 | 71.69 | 75.83 | 75.98 | 70.43 | 73.10 | 97.96 | 93.81 | 95.84 | 95.20 | 97.69 | 96.43  
KAPT*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT [17] | 78.41 | 70.52 | 74.26 | 71.10 | 65.20 | 68.02 | 97.10 | 93.53 | 95.28 | 93.13 | 96.53 | 94.80  
ProDA [24] | 81.56 | 72.30 | 76.65 | 76.66 | 70.54 | 73.47 | 97.74 | 94.36 | 96.02 | 95.43 | 97.76 | 96.58  
MaPLe [18] | 82.28 | 75.14 | 78.55 | 75.40 | 70.32 | 72.72 | 98.27 | 93.23 | 95.68 | 95.43 | 97.83 | 96.62  
RPO [23] | 81.13 | 75.00 | 77.78 | 76.60 | 71.57 | 74.00 | 97.97 | 94.37 | 96.03 | 94.63 | 97.50 | 96.05  
PSRC [19] | 84.26 | 76.10 | 79.97 | 77.60 | 70.73 | 74.01 | 98.10 | 94.03 | 96.02 | 95.33 | 97.30 | 96.30  
LLaMP | 85.16 | 77.71 | 81.27 | 77.99 | 71.27 | 74.48 | 98.45 | 95.85 | 97.13 | 96.31 | 97.74 | 97.02  
ΔΔ\Deltaroman_Δ w.r.t. PSRC | +0.90 | +1.61 | +1.30 | +0.39 | +0.54 | +0.47 | +0.35 | +1.82 | +1.11 | +0.98 | +0.44 | +0.72  
  
Method |  StanfordCars [21] |  Flowers102[26] |  Food101 [3] |  FGVCAircraft [25]  
---|---|---|---|---  
Base | Novel | HM | Base | Novel | HM | Base | Novel | HM | Base | Novel | HM  
CLIP [32] | 63.37 | 74.89 | 68.65 | 72.08 | 77.80 | 74.83 | 90.10 | 91.22 | 90.66 | 27.19 | 36.29 | 31.09  
CoOp [49] | 78.12 | 60.40 | 68.13 | 97.60 | 59.67 | 74.06 | 88.33 | 82.26 | 85.19 | 40.44 | 22.30 | 28.75  
CoCoOp [48] | 70.49 | 73.59 | 72.01 | 94.87 | 71.75 | 81.71 | 90.70 | 91.29 | 90.99 | 33.41 | 23.71 | 27.74  
KAPT*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT [17] | 69.47 | 66.20 | 67.79 | 95.00 | 71.20 | 81.40 | 86.13 | 87.06 | 86.59 | 29.67 | 28.73 | 29.19  
ProDA [24] | 72.94 | 74.00 | 73.47 | 95.92 | 72.46 | 82.56 | 90.71 | 92.05 | 91.38 | 37.44 | 35.61 | 36.50  
MaPLe [18] | 74.70 | 71.20 | 72.91 | 97.70 | 68.68 | 80.66 | 90.30 | 88.57 | 89.43 | 36.90 | 34.13 | 35.46  
RPO [23] | 73.87 | 75.53 | 74.69 | 94.13 | 76.67 | 84.50 | 90.33 | 90.83 | 90.58 | 37.33 | 34.20 | 35.70  
PSRC [19] | 78.27 | 74.97 | 76.58 | 98.07 | 76.50 | 85.95 | 90.67 | 91.53 | 91.10 | 42.73 | 37.87 | 40.15  
LLaMP | 81.56 | 74.54 | 77.89 | 97.82 | 77.40 | 86.42 | 91.05 | 91.93 | 91.49 | 47.30 | 37.61 | 41.90  
ΔΔ\Deltaroman_Δ w.r.t. PSRC | +3.29 | -0.43 | +1.31 | -0.25 | +0.90 | +0.47 | +0.38 | +0.40 | +0.39 | +4.57 | -0.26 | +1.75  
  
Method |  SUN397 [45] |  DTD [7] |  EuroSAT [11] |  UCF101 [37]  
---|---|---|---|---  
Base | Novel | HM | Base | Novel | HM | Base | Novel | HM | Base | Novel | HM  
CLIP [32] | 69.36 | 75.35 | 72.23 | 53.24 | 59.90 | 56.37 | 56.48 | 64.05 | 60.03 | 70.53 | 77.50 | 73.85  
CoOp [49] | 80.60 | 65.89 | 72.51 | 79.44 | 41.18 | 54.24 | 92.19 | 54.74 | 68.69 | 84.69 | 56.05 | 67.46  
CoCoOp [48] | 79.74 | 76.86 | 78.27 | 77.01 | 56.00 | 64.85 | 87.49 | 60.04 | 71.21 | 82.33 | 73.45 | 77.64  
KAPT*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT [17] | 79.40 | 74.33 | 76.78 | 75.97 | 58.30 | 65.97 | 84.80 | 67.57 | 75.21 | 80.83 | 67.10 | 73.33  
ProDA [24] | 80.82 | 78.70 | 79.75 | 80.36 | 59.18 | 68.16 | 94.07 | 73.23 | 82.35 | 83.00 | 78.66 | 80.77  
MaPLe [18] | 78.47 | 76.93 | 77.79 | 80.67 | 56.48 | 66.44 | 83.90 | 66.00 | 73.88 | 85.23 | 71.97 | 78.04  
RPO [23] | 80.60 | 77.80 | 79.18 | 76.70 | 62.13 | 68.61 | 86.63 | 68.97 | 76.79 | 83.67 | 75.43 | 79.34  
PSRC [19] | 82.67 | 78.47 | 80.52 | 83.37 | 62.97 | 71.75 | 92.90 | 73.90 | 82.32 | 87.10 | 78.80 | 82.74  
LLaMP | 83.41 | 79.90 | 81.62 | 83.49 | 64.49 | 72.77 | 91.93 | 83.66 | 87.60 | 87.13 | 80.66 | 83.77  
ΔΔ\Deltaroman_Δ w.r.t. PSRC | +0.74 | +1.43 | +1.10 | +0.12 | +1.52 | +1.02 | -0.97 | +9.76 | +5.28 | +0.03 | +1.86 | +1.03  
  
Table 1: Comparison with state-of-the-art methods on base-to-novel generalization. LLaMP shows strong generalization results over previous approaches on 11 image classification tasks. Absolute gains over PSRC are indicated in blue. *{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPTKAPT is trained with ViT-B/32 image encoder instead of ViT-B/16.

###  4.1 Experiment Setup

Datasets. Similar to previous work [19, 18, 48], in our study, we evaluate LLaMP performance over a spectrum of classification tasks with 11 datasets, including ImageNet [8] and Caltech101 [10] for generic image classification, OxfordPets [29], StanfordCars [21], Flowers102 [26], Food101 [3], and FGVCAircraft [25] for fine-grained classification, SUN397 [45] for scene recognition, UCF101 [37] for action recognition, DTD [7] for texture classification, and EuroSAT [11] for satellite image recognition.

| 16-Shot Classification  
---|---  
|  _Average_ |  ImageNet [8] |  Caltech [10] |  Pets [29] |  Cars [21] |  Flowers [26] |  Food [3] |  Aircraft [25] |  SUN397 [45] |  DTD [7] |  EuroSAT [11] |  UCF101 [37]  
CLIP [32] | 78.79 | 67.31 | 95.43 | 85.34 | 80.44 | 97.37 | 82.90 | 45.36 | 73.28 | 69.96 | 87.21 | 82.11  
CoOp [49] | 79.89 | 71.87 | 95.57 | 91.87 | 83.07 | 97.07 | 84.20 | 43.40 | 74.67 | 69.87 | 84.93 | 82.23  
CoCoOp [48] | 74.90 | 70.83 | 95.16 | 93.34 | 71.57 | 87.84 | 87.25 | 31.21 | 72.15 | 63.04 | 73.32 | 78.14  
MaPLe [18] | 81.79 | 72.33 | 96.00 | 92.83 | 83.57 | 97.00 | 85.33 | 48.40 | 75.53 | 71.33 | 92.33 | 85.03  
PSRC [19] | 82.87 | 73.17 | 96.07 | 93.67 | 83.83 | 97.60 | 87.50 | 50.83 | 77.23 | 72.73 | 92.43 | 86.47  
LLaMP | 83.81 | 73.49 | 97.08 | 94.21 | 86.07 | 98.06 | 87.62 | 56.07 | 77.02 | 74.17 | 91.31 | 86.84  
Table 2: Few shot classification results with 16 shots.

Scenarios & Metrics. We evaluate LLaMP on two typical low-shot scenarios: zero-shot base-to-novel generalization and few-shot image classification. In zero-shot base-to-novel generalization, the base classes are seen during training, while the novel classes are unseen. We measure models performance through accuracies of base and novel classes, and the harmonic mean of the two. For few-shot classification, we assess the accuracy with 16 shots per class.

Implementation Details. We build LLaMP through the PyTorch [30] framework. All models are trained with 2 NVIDIA A100 40GB GPUs. For LLaMP, we adopt LLaMA2-7B [39] as the language model 𝒟𝒟\mathcal{D}caligraphic_D, and ViT-B/16 [9] as the image encoder, following [19, 18, 48, 49]. On the text side, we set prompt learning depth to 9. To tune the vision encoder, we adopt the hybrid tuning scheme which performs deep prompt learning on the first 6 layers and LoRA on the rest. Similar to [13], LoRA is applied to the Query and Value projection layers inside attention modules. The number of 𝒑lsubscript𝒑𝑙\bm{p}_{l}bold_italic_p start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT prompts, K𝐾Kitalic_K, is set to 16. We set a global learning rate of 2⁢e-⁢42e-42\text{e-}42 e- 4 with a batch size of 8. The learning rate of LoRA modules is set to 2⁢e-⁢52e-52\text{e-}52 e- 5. λt,λvsubscript𝜆𝑡subscript𝜆𝑣\lambda_{t},\lambda_{v}italic_λ start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT , italic_λ start_POSTSUBSCRIPT italic_v end_POSTSUBSCRIPT and λd⁢i⁢s⁢tsubscript𝜆𝑑𝑖𝑠𝑡\lambda_{dist}italic_λ start_POSTSUBSCRIPT italic_d italic_i italic_s italic_t end_POSTSUBSCRIPT are set to 25, 10 and 2.5, respectively.

###  4.2 Quantitative Evaluation

Zero-Shot Base-to-Novel Generalization. LLaMP outperforms existing state-of-the-art prompt learning methods on most metrics of 11 classification datasets in the base-to-novel generalization benchmark. As shown in Tab. 1, compared to the latest model PSRC [19], LLaMP achieves average gains of 0.90% in base accuracy, 1.61% in novel accuracy, and 1.30% in harmonic mean on average. Moreover, LLaMP consistently achieves higher harmonic means (HM) compared to other models. These improvements indicate that our approach better balances performance on base and novel data, thus achieving stronger generalization compared to the prior prompt learning techniques.

In particular, LLaMP excels in fine-grained datasets requiring detailed analysis. On FGVCAircraft, LLaMP surpasses PSRC by 4.57% on base accuracy and 1.75% on HM, highlighting its strong understanding of detailed aircraft features. Furthermore, on EuroSAT, LLaMP achieves improvements of 9.76% and 5.28% on novel accuracy and HM, respectively. We also observe similar performance gains on StanfordCars, where LLaMP outperformns by 3.29% on base accuracy and 1.31% on HM. The information embedded in LLM enables LLaMP to capture and utilize the rich semantic information necessary for distinguishing between closely related categories.

Few-Shot Classification. LLaMP also achieves improvements across these classification datasets in few-shot classification tasks. As in Tab.2, with an average classification accuracy of 83.81%. Notably, on FGVCAircraft and StanfordCars, LLaMP shows a significant improvement over PSRC, further demonstrating that the knowledge from language models benefits the recognition of fine-grained object categories, which aligns with our observation on zero-shot base-to-novel generalization. Moreover, on DTD, where MaPLe and PSRC achieve around 72% accuracy, LLaMP achieves a higher accuracy of 74.17%, underscoring its ability to recognize textures.

###  4.3 Ablation Study

Method | LLM | Base | Novel | HM  
---|---|---|---|---  
CLIP |  | 69.34 | 74.22 | 71.70  
✓ | 70.95 | 74.93 | 72.79  
LLaMP |  | 82.21 | 76.44 | 79.22  
✓ | 85.16 | 77.71 | 81.27  
Table 3: Ablation study on the LLM Knowledge.

LP | QO | KV | FFN | % | Base | Novel | HM  
---|---|---|---|---|---|---|---  
✓ |  |  |  | .03 | 85.00 | 77.29 | 80.96  
✓ | ✓ | ✓ |  | 33 | 85.20 | 77.45 | 81.14  
✓ | ✓ |  | ✓ | 83 | 85.05 | 77.73 | 81.22  
✓ | ✓ | ✓ | ✓ | 100 | 85.23 | 77.56 | 81.21  
✓ | ✓ |  |  | 17 | 85.16 | 77.71 | 81.27  
  
Table 4: Ablation study on the Training Strategy. “%” indicates the ratio of parameters trained compared to fully tuning a layer.

Is the knowledge from LLM helping? In Tab. 3, we show that the knowledge from LLM benefits in both ways: Without training, performance of ordinary CLIP model can be improved by introducing noun phrases; The LLaMP framework shows further improvement after training.

Noun phrases are parsed from the LLM’s response the prompt of “Describe [STH]”. We then use the template, “A photo of [STH] with [NP]” to generate the NP-augmented text embedding for CLIP. We take the average of all augmented embeddings for classification. In Tab. 3 we show that even ordinary CLIP can benefit from incorporating LLMs’ knowledge.

Furthermore, the comparison between LLaMP and LLaMP without the LLM indicates that merely integrating LoRA [13] to the vision encoder is not beneficial. The “LLaMP without LLM” is essentially an ordinary prompting learning model plus LoRAs in the vision encoder. We show that the improved vision encoding capacity only benefits when the quality of text embeddings s are enhanced by incorporating LLMs’ knowledge through LLaMP.

Decoder Training strategy. We categorize trainable parameters of 𝒟Nsubscript𝒟𝑁\mathcal{D}_{N}caligraphic_D start_POSTSUBSCRIPT italic_N end_POSTSUBSCRIPT into four groups: learnable prompts (LP), Query and Output projections (QO), Key and Value projections (KV), and the feed-forward network (FFN). Tab. 4 indicates LLaMP can achieve desirable results by just learning the prompts of 𝒟𝒟\mathcal{D}caligraphic_D. One step further, adding QO into optimization achieves the best performance. Although other setups introduce much more trainable parameters, they can not surpass the “LP + QO” strategy.

Effect of Textual Priors. We study the effect of pre-generated textual priors on LLaMP. We compare three different approaches: without textual priors, using plain responses as the prior, and LLaMP, which takes parsed noun phrases. Tab. 5 shows that LLaMP can achieve over 81% on HM without pre-generated priors, while adding parsed noun phrases as textual priors further pushes the HM to 81.27%.

Method | Priors | Base | Novel | HM  
---|---|---|---|---  
LLaMP | ✗ | 84.90 | 77.59 | 81.08  
Plain | 85.26 | 77.56 | 81.22  
NP | 85.16 | 77.71 | 81.27  
Table 5: Ablation Study on Pre-generated Text Priors. ✗ refers to “without textual priors” and NP stands for noun phrases. Method | Base | Novel | HM  
---|---|---|---  
LLM Only | 81.74 | 35.82 | 49.81  
LLaMP | 85.16 | 77.71 | 81.27  
Table 6: The CLIP text encoder helps adaptation. Scheme | Base | Novel | HM  
---|---|---|---  
Prompt ×\times× 9 | 84.67 | 77.28 | 80.81  
LoRA ×\times× 12 | 84.89 | 77.27 | 80.90  
Prompt ×6absent6\times 6× 6 \+ LoRA ×6absent6\times 6× 6 | 85.16 | 77.71 | 81.27  
Table 7: Study on Vision Tuning Scheme. Our hybrid design achieves the best performance.

CLIP as the bridge. One may wonder if it is possible to replace CLIP text encoder with a large language model. Here, we study two setups: i) LLM as encoder, which treats the output of the language model, 𝒉~lsubscriptbold-~𝒉𝑙\bm{\tilde{h}}_{l}overbold_~ start_ARG bold_italic_h end_ARG start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT as the text feature; ii) LLaMP, which treat 𝒉~lsubscriptbold-~𝒉𝑙\bm{\tilde{h}}_{l}overbold_~ start_ARG bold_italic_h end_ARG start_POSTSUBSCRIPT italic_l end_POSTSUBSCRIPT as part of the text input prompt. Tab. 6 reveals that relying solely on the LLM results in poor accuracy for novel categories. This supports our hypothesis that aligning LLMs with vision encoders generally requires a more extensive dataset. Furthermore, LLaMP’s design significantly improve the novel accuracy by 40%.

Vision Training Strategy. As ViT-16/B has 12 transformer layers, we compare different vision training strategies within LLaMP in Tab. 7. Apart from the default hybrid scheme, we evaluate setups including prompt learning at first 9 layers (P9), a similar setup to PSRC [19], and LoRA [13] in all 12 layers. The results suggest that the scheme leverages the strengths of both prompt learning and LoRA, addressing potential bottlenecks in the vision encoder and enhancing overall performance in LLaMP.

Number of LLM Prompts. We vary the number of LLM prompts and study their effects on LLaMP. As in Fig. 3, using 16 prompts optimizes the LLM’s capabilities, achieving the highest harmonic mean at 81.27%.

![Refer to caption](x4.png) Figure 3: Effect of LLM Prompts on Harmonic Mean. 16 prompts achieve the most balanced performance.

Image | Noun Phrases | Heatmap  
---|---|---  
Classname: An-12  
![Refer to caption](extracted/5279485/figures/vis/FGVCAircraft_1218_rgb.png) |  four engines, turboprop aircraft, large vertical fin |  ![Refer to caption](extracted/5279485/figures/vis/FGVCAircraft_1218_vis.png)  
Classname: Industrial Buildings  
![Refer to caption](extracted/5279485/figures/vis/EuroSAT_4115_rgb.png) |  a cluster, rectangular structures, flat roofs, straight lines |  ![Refer to caption](extracted/5279485/figures/vis/EuroSAT_4115_vis.png)  
  
Figure 4: Visualization of LLaMP Predictions by GradCAM [35]

Visualizations. In Fig. 4, we visualize the gradient heatmap of input images from FGVCAircraft and EuroSAT, through GradCAM [35]. The figure shows that, with the knowledge from an LLM, LLaMP is able to capture distinctive features that matches LLM’s description.

##  5 Conclusion

Our study shows that the encyclopedic knowledge from LLMs is beneficial for low-shot image classification as extra class-specific information. To leverage such knowledge, we propose LLaMP, a framework that adapts LLMs as prompt learners for the CLIP model. Over two common low-shot scenarios: zero-shot generalization and few-shot learning, LLaMP demonstrates notable improvements compared with previous state-of-the-arts on a spectrum of datasets.

Limitations. While LLaMP reveals an effective way in leveraging LLMs’ knowledge, both modalities, vision and language, only interact at the finest feature level. Given the broader LLM-aided knowledge from the language side, the performance can be potentially further improved by introducing language priors at earlier vision encoding stages.

## References

  * Almazrouei et al. [2023] Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Merouane Debbah, Etienne Goffinet, Daniel Heslow, Julien Launay, Quentin Malartic, et al.  Falcon-40b: an open large language model with state-of-the-art performance.  Technical report, Technical report, Technology Innovation Institute, 2023. 
  * Anil et al. [2023] Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al.  Palm 2 technical report.  _arXiv preprint arXiv:2305.10403_ , 2023. 
  * Bossard et al. [2014] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool.  Food-101–mining discriminative components with random forests.  In _ECCV_ , pages 446–461. Springer, 2014. 
  * Chen et al. [2021] Shiming Chen, Wenjie Wang, Beihao Xia, Qinmu Peng, Xinge You, Feng Zheng, and Ling Shao.  Free: Feature refinement for generalized zero-shot learning.  In _ICCV_ , pages 122–131, 2021. 
  * Chiang et al. [2023] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023. 
  * Chowdhery et al. [2022] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al.  Palm: Scaling language modeling with pathways.  _arXiv preprint arXiv:2204.02311_ , 2022. 
  * Cimpoi et al. [2014] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi.  Describing textures in the wild.  In _CVPR_ , pages 3606–3613, 2014. 
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _CVPR_ , pages 248–255. Ieee, 2009. 
  * Dosovitskiy et al. [2021] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  In _ICLR_ , 2021. 
  * Fei-Fei et al. [2004] Li Fei-Fei, Rob Fergus, and Pietro Perona.  Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories.  In _CVPR workshop_ , pages 178–178. IEEE, 2004. 
  * Helber et al. [2019] Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth.  Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification.  _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , 12(7):2217–2226, 2019. 
  * Honnibal et al. [2020] Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd.  spaCy: Industrial-strength Natural Language Processing in Python.  2020\. 
  * Hu et al. [2021] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_ , 2021. 
  * Jia et al. [2021] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig.  Scaling up visual and vision-language representation learning with noisy text supervision.  In _ICML_ , pages 4904–4916. PMLR, 2021. 
  * Jia et al. [2022] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim.  Visual prompt tuning.  In _European Conference on Computer Vision_ , pages 709–727. Springer, 2022. 
  * Kampffmeyer et al. [2019] Michael Kampffmeyer, Yinbo Chen, Xiaodan Liang, Hao Wang, Yujia Zhang, and Eric P Xing.  Rethinking knowledge graph propagation for zero-shot learning.  In _CVPR_ , pages 11487–11496, 2019. 
  * Kan et al. [2023] Baoshuo Kan, Teng Wang, Wenpeng Lu, Xiantong Zhen, Weili Guan, and Feng Zheng.  Knowledge-aware prompt tuning for generalizable vision-language models.  In _ICCV_ , pages 15670–15680, 2023. 
  * Khattak et al. [2023a] Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan.  Maple: Multi-modal prompt learning.  In _CVPR_ , pages 19113–19122, 2023a. 
  * Khattak et al. [2023b] Muhammad Uzair Khattak, Syed Talal Wasim, Muzammal Naseer, Salman Khan, Ming-Hsuan Yang, and Fahad Shahbaz Khan.  Self-regulating prompts: Foundational model adaptation without forgetting.  In _ICCV_ , pages 15190–15200, 2023b. 
  * Kipf and Welling [2017] Thomas N. Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  In _5th ICLR, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net, 2017. 
  * Krause et al. [2013] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei.  3d object representations for fine-grained categorization.  In _ICCV workshops_ , pages 554–561, 2013. 
  * Lampert et al. [2013] Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling.  Attribute-based classification for zero-shot visual object categorization.  _TPAMI_ , 36(3):453–465, 2013. 
  * Lee et al. [2023] Dongjun Lee, Seokwon Song, Jihee Suh, Joonmyeong Choi, Sanghyeok Lee, and Hyunwoo J Kim.  Read-only prompt optimization for vision-language few-shot learning.  In _ICCV_ , pages 1401–1411, 2023. 
  * Lu et al. [2022] Yuning Lu, Jianzhuang Liu, Yonggang Zhang, Yajing Liu, and Xinmei Tian.  Prompt distribution learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 5206–5215, 2022. 
  * Maji et al. [2013] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi.  Fine-grained visual classification of aircraft.  _arXiv preprint arXiv:1306.5151_ , 2013. 
  * Nilsback and Zisserman [2008] Maria-Elena Nilsback and Andrew Zisserman.  Automated flower classification over a large number of classes.  In _2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing_, pages 722–729. IEEE, 2008. 
  * OpenAI [2023a] OpenAI.  Chatgpt, 2023a.  Available at <https://openai.com/chatgpt>. 
  * OpenAI [2023b] OpenAI.  Gpt-4 technical report, 2023b. 
  * Parkhi et al. [2012] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar.  Cats and dogs.  In _CVPR_ , pages 3498–3505. IEEE, 2012. 
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  _NeurIPS_ , pages 8026–8037, 2019. 
  * Pope et al. [2023] Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean.  Efficiently scaling transformer inference.  _Proceedings of Machine Learning and Systems_ , 5, 2023. 
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _ICML_ , pages 8748–8763. PMLR, 2021. 
  * Rasheed et al. [2023] Hanoona Rasheed, Muhammad Uzair Khattak, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan.  Fine-tuned clip models are efficient video learners.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 6545–6554, 2023. 
  * Reed et al. [2016] Scott Reed, Zeynep Akata, Honglak Lee, and Bernt Schiele.  Learning deep representations of fine-grained visual descriptions.  In _CVPR_ , pages 49–58, 2016. 
  * Selvaraju et al. [2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _ICCV_ , pages 618–626, 2017. 
  * Socher et al. [2013] Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D Manning, and Andrew Y Ng.  Zero-shot learning through cross-modal transfer.  _arXiv preprint arXiv:1301.3666_ , 2013. 
  * Soomro et al. [2012] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah.  Ucf101: A dataset of 101 human actions classes from videos in the wild.  _arXiv preprint arXiv:1212.0402_ , 2012. 
  * Touvron et al. [2023a] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ , 2023a. 
  * Touvron et al. [2023b] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_ , 2023b. 
  * Wang et al. [2018] Xiaolong Wang, Yufei Ye, and Abhinav Gupta.  Zero-shot recognition via semantic embeddings and knowledge graphs.  In _CVPR_ , pages 6857–6866, 2018. 
  * Wang et al. [2022a] Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al.  Dualprompt: Complementary prompting for rehearsal-free continual learning.  In _European Conference on Computer Vision_ , pages 631–648. Springer, 2022a. 
  * Wang et al. [2022b] Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister.  Learning to prompt for continual learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 139–149, 2022b. 
  * Wolf et al. [2020] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online, 2020. Association for Computational Linguistics. 
  * Xian et al. [2018] Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata.  Feature generating networks for zero-shot learning.  In _CVPR_ , pages 5542–5551, 2018. 
  * Xiao et al. [2016] Jianxiong Xiao, Krista A Ehinger, James Hays, Antonio Torralba, and Aude Oliva.  Sun database: Exploring a large collection of scene categories.  _IJCV_ , 119(1):3–22, 2016. 
  * Zhang et al. [2017] Li Zhang, Tao Xiang, and Shaogang Gong.  Learning a deep embedding model for zero-shot learning.  In _CVPR_ , pages 2021–2030, 2017. 
  * Zhang et al. [2022] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al.  Opt: Open pre-trained transformer language models.  _arXiv preprint arXiv:2205.01068_ , 2022. 
  * Zhou et al. [2022a] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu.  Conditional prompt learning for vision-language models.  In _CVPR_ , pages 16816–16825, 2022a. 
  * Zhou et al. [2022b] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu.  Learning to prompt for vision-language models.  _IJCV_ , 130(9):2337–2348, 2022b. 
  * Zhu et al. [2018] Yizhe Zhu, Mohamed Elhoseiny, Bingchen Liu, Xi Peng, and Ahmed Elgammal.  A generative adversarial approach for zero-shot learning from noisy texts.  In _CVPR_ , pages 1004–1013, 2018. 



## Appendix Overview

Here, we provide extra discussions that did not fit in the main paper due to the space limitation, including i) Ablation study on the textual augmentation described in Sec 3.3; ii) Few-shot classification results on 8/4/2/1 shots with comparisons with previous methods.

## Ablation on Textual Augmentations

As discussed in Sec 3.3, we perform textual augmentations in two steps: i) When computing 𝒈^^𝒈\hat{\bm{g}}over^ start_ARG bold_italic_g end_ARG, we replace the original template “A photo of [STH]”, with “A photo of [STH] with [NP]”, thereby enriching the descriptive content with noun phrases extracted from LLM responses; ii) We create new LLM prompt templates similar to “In one sentence, describe the distinctive appearance of [STH]” through GPT-4 [28], and average the scores for final prediction.

We show ablation study results in Tab. 8, with TA1 and TA2 referring to the step i) and ii) mentioned above. Results show that, even without textual augmentations, LLaMP still outperforms PSRC, the previous state-of-the-art, by 0.68% on base accuracy, 0.94% on novel accuracy and 0.83% on the harmonic mean. Moreover, we observe that both augmentation steps further improve the performance of LLaMP. More specifically, TA1 improves the HM by 0.35% while TA2 brings in another boost of 0.12%.

Method | TA1 | TA2 | Base | Novel | HM  
---|---|---|---|---|---  
PSRC [19] |  |  | 84.26 | 76.10 | 79.97  
LLaMP |  |  | 84.94 | 77.04 | 80.80  
| ✓ | 84.78 | 77.31 | 80.86  
✓ |  | 85.16 | 77.50 | 81.15  
✓ | ✓ | 85.16 | 77.71 | 81.27  
Table 8: Ablation study on textual augmentations.

## Few-shot Classification

In addition to the 16-shot classification result reported in Tab. 2, we present few-shot classification results with with 8/4/2/1 numbers of shots in Tab. 9 and compare LLaMP against previous baseline models.

Results in Tab. 9 show that LLaMP outperforms previous SOTAs under all settings, on average of all 11 benchmarks, with 0.88% improvement with 8 shots. In particular, we observe that LLaMP surpasses PSRC [19] consistently on FGVCAircraft (Aircraft) [25] and Food [3] with all numbers of shots. Such observation aligns with our argument in the main paper that the knowledge from LLMs provides richer semantic information for fine-grained classification.

| 8-Shot Classification  
---|---  
|  _Average_ |  ImageNet [8] |  Caltech [10] |  Pets [29] |  Cars [21] |  Flowers [26] |  Food [3] |  Aircraft [25] |  SUN397 [45] |  DTD [7] |  EuroSAT [11] |  UCF101 [37]  
CLIP [32] | 74.47 | 62.23 | 93.41 | 78.36 | 73.67 | 96.10 | 79.79 | 39.35 | 69.08 | 63.46 | 84.43 | 79.34  
CoOp [49] | 76.98 | 70.63 | 94.37 | 91.27 | 79.30 | 94.97 | 82.67 | 39.00 | 71.53 | 64.77 | 78.07 | 80.20  
CoCoOp [48] | 72.96 | 70.63 | 95.04 | 93.45 | 70.44 | 84.30 | 86.97 | 26.61 | 70.84 | 58.89 | 68.21 | 77.14  
MaPLe [18] | 78.89 | 70.30 | 95.20 | 92.57 | 79.47 | 95.80 | 83.60 | 42.00 | 73.23 | 66.50 | 87.73 | 81.37  
PSRC [19] | 80.69 | 72.33 | 95.67 | 93.50 | 80.97 | 96.27 | 86.90 | 43.27 | 75.73 | 69.87 | 88.80 | 84.30  
LLaMP | 81.57 | 72.30 | 96.57 | 93.69 | 82.15 | 96.20 | 87.39 | 47.48 | 75.18 | 71.14 | 91.15 | 84.06  
  
| 4-Shot Classification  
---|---  
|  _Average_ |  ImageNet [8] |  Caltech [10] |  Pets [29] |  Cars [21] |  Flowers [26] |  Food [3] |  Aircraft [25] |  SUN397 [45] |  DTD [7] |  EuroSAT [11] |  UCF101 [37]  
CLIP [32] | 68.01 | 54.85 | 92.05 | 71.17 | 63.38 | 92.02 | 73.19 | 32.33 | 63.00 | 55.71 | 77.09 | 73.28  
CoOp [49] | 74.02 | 68.73 | 94.40 | 92.57 | 74.47 | 92.17 | 84.47 | 30.83 | 69.97 | 58.70 | 70.80 | 77.10  
CoCoOp [48] | 71.21 | 70.39 | 94.98 | 92.81 | 69.39 | 78.40 | 86.88 | 24.79 | 70.21 | 55.04 | 65.56 | 74.82  
MaPLe [18] | 75.37 | 67.70 | 94.43 | 91.90 | 75.30 | 92.67 | 81.77 | 34.87 | 70.67 | 61.00 | 84.50 | 78.47  
PSRC [19] | 78.35 | 71.07 | 95.27 | 93.43 | 77.13 | 93.87 | 86.17 | 37.47 | 74.00 | 65.53 | 86.30 | 81.57  
LLaMP | 78.83 | 71.37 | 95.84 | 93.61 | 76.79 | 93.96 | 87.17 | 40.02 | 74.05 | 66.37 | 86.16 | 81.80  
  
| 2-Shot Classification  
---|---  
|  _Average_ |  ImageNet [8] |  Caltech [10] |  Pets [29] |  Cars [21] |  Flowers [26] |  Food [3] |  Aircraft [25] |  SUN397 [45] |  DTD [7] |  EuroSAT [11] |  UCF101 [37]  
CLIP [32] | 57.98 | 44.88 | 89.01 | 58.37 | 50.28 | 85.07 | 61.51 | 26.41 | 53.70 | 40.76 | 61.98 | 65.78  
CoOp [49] | 70.65 | 67.07 | 93.07 | 89.80 | 70.50 | 87.33 | 84.40 | 26.20 | 66.53 | 53.60 | 65.17 | 73.43  
CoCoOp [48] | 67.65 | 69.78 | 94.82 | 92.64 | 68.37 | 75.79 | 86.22 | 15.06 | 69.03 | 52.17 | 46.74 | 73.51  
MaPLe [18] | 72.58 | 65.10 | 93.97 | 90.87 | 71.60 | 88.93 | 81.47 | 30.90 | 67.10 | 55.50 | 78.30 | 74.60  
PSRC [19] | 75.29 | 69.77 | 94.53 | 92.50 | 73.40 | 91.17 | 85.70 | 31.70 | 71.60 | 59.97 | 79.37 | 78.50  
LLaMP | 75.89 | 70.12 | 95.66 | 92.75 | 72.20 | 89.16 | 86.33 | 33.41 | 72.64 | 61.29 | 81.71 | 79.56  
  
| 1-Shot Classification  
---|---  
|  _Average_ |  ImageNet [8] |  Caltech [10] |  Pets [29] |  Cars [21] |  Flowers [26] |  Food [3] |  Aircraft [25] |  SUN397 [45] |  DTD [7] |  EuroSAT [11] |  UCF101 [37]  
CLIP [32] | 45.83 | 32.13 | 79.88 | 44.06 | 35.66 | 69.74 | 43.96 | 19.61 | 41.58 | 34.59 | 49.23 | 53.66  
CoOp [49] | 67.56 | 66.33 | 92.60 | 90.37 | 67.43 | 77.53 | 84.33 | 21.37 | 66.77 | 50.23 | 54.93 | 71.23  
CoCoOp [48] | 66.79 | 69.43 | 93.83 | 91.27 | 67.22 | 72.08 | 85.65 | 12.68 | 68.33 | 48.54 | 55.33 | 70.30  
MaPLe [18] | 69.27 | 62.67 | 92.57 | 89.10 | 66.60 | 83.30 | 80.50 | 26.73 | 64.77 | 52.13 | 71.80 | 71.83  
PSRC [19] | 72.32 | 68.13 | 93.67 | 92.00 | 69.40 | 85.93 | 84.87 | 27.67 | 69.67 | 56.23 | 73.13 | 74.80  
LLaMP | 72.42 | 69.12 | 94.59 | 91.91 | 70.02 | 84.03 | 85.83 | 30.39 | 69.69 | 54.98 | 70.36 | 75.72  
  
Table 9: Few shot classification results with 8/4/2/1 shots. All numbers, excepts ours, are obtained from [19].



url: https://promptengineering.org/zero-shot-prompting-a-powerful-technique-for-llms/
title: Zero-Shot Prompting: A Powerful Technique for LLMs
scrape_date: 2025-03-17T12:18:26.655501
content_type: markdown
hash: 27243b6fcc5a95e49cff8090c8944de2962d527b7438abeb30afca3188cb799e
content:
[ ![Prompt Engineering Institute ](https://digitalpress.fra1.cdn.digitaloceanspaces.com/75xl145/2024/03/prompt-engineering-and-ai-institute-2.png) ](https://promptengineering.org)

  * [ Learn ](https://promptengineering.org/Learn/)
  * [ Free Course ](https://promptengineering.org/prompt-engineering-course/)
  * [ About ](https://promptengineering.org/about/)
  * [ Contact ](https://promptengineering.org/contact-us/)
  * [ Community ](https://promptengineer.net/communities/groups/prompt-engineering-community/private-group)



_ _

  * [ Learn ](https://promptengineering.org/Learn/)
  * [ Free Course ](https://promptengineering.org/prompt-engineering-course/)
  * [ About ](https://promptengineering.org/about/)
  * [ Contact ](https://promptengineering.org/contact-us/)
  * [ Community ](https://promptengineer.net/communities/groups/prompt-engineering-community/private-group)



  * Sign up
  * [ Contact Us ](https://promptengineering.org/contact-us/)



  * [0-Shot](/tag/0-shot/)
  * [Lesson](/tag/lesson/)
  * [Prompting](/tag/prompting/)

Jul 23, 2024

#  Zero-Shot Prompting: A Powerful Technique for LLMs 

A look at zero-shot prompting, a technique that enables large language models to perform tasks without explicit training data. Explore its benefits, limitations, best practices, and real-world applications.

  * [ ![](https://digitalpress.fra1.cdn.digitaloceanspaces.com/75xl145/2023/10/SR-Window-1.png) Sunil Ramlochan ](/author/sunil/)



Table of Contents _ _ _ _

## 1\. Introduction to Zero-Shot Prompting

### 1.1 What is Zero-Shot Prompting?

Zero-shot prompting exemplified the progress in natural language processing (NLP) and the advent of increasingly sophisticated large language models (LLMs). In essence, it's a paradigm where an LLM, trained on a massive dataset of text and code, is able to perform a task without prior task-specific examples or demonstrations. Unlike traditional machine learning approaches that rely heavily on labeled data for specific tasks, zero-shot prompting allows LLMs to generalize their knowledge and understanding to new and unseen challenges.

### 1.2 Capabilities of Modern LLMs

Modern LLMs, such as GPT-4 Turbo and Claude 3, exhibit an exceptional ability to comprehend and generate human-like text. This proficiency stems from their extensive training on diverse datasets, encompassing a vast spectrum of topics and writing styles. Through this training, they develop an intricate understanding of language structure, semantics, and even nuances like sentiment and tone. This inherent ability to decipher and process language empowers them to excel in zero-shot prompting scenarios.

### 1.3 Benefits of Zero-Shot Prompting

Zero-shot prompting presents several key advantages:

  * **Efficiency:**  It eliminates the need for time-consuming and often expensive data labeling processes, as the model is not reliant on task-specific examples.
  * **Flexibility:**  LLMs can readily adapt to new tasks and domains without requiring retraining, making them highly versatile tools.
  * **Scalability:**  Zero-shot prompting facilitates the application of LLMs to a wide array of NLP tasks, even those with limited training data.



## 2\. Understanding Zero-Shot Prompting

### 2.1 How Zero-Shot Prompting Works

Zero-shot prompting capitalizes on the vast knowledge encoded within the parameters of large language models during their pretraining phase. These models are exposed to massive datasets containing text and code, enabling them to learn patterns, relationships, and representations of various concepts. When presented with a zero-shot prompt, the LLM taps into this internalized knowledge to deduce the task at hand and generate a response.

Consider this analogy: imagine an individual who has read extensively on a variety of subjects but hasn't been explicitly taught how to perform a specific task, like baking a cake. When given a detailed recipe (the zero-shot prompt), this person can leverage their existing knowledge and infer the necessary steps to bake the cake successfully. Similarly, an LLM utilizes its vast knowledge base to interpret the prompt and formulate a coherent response, even without prior task-specific demonstrations.

### 2.2 Illustrative Example of Zero-Shot Prompting (e.g., Sentiment Analysis)

Let's examine a practical example in the realm of sentiment analysis. Suppose we want an LLM to determine the sentiment expressed in the following sentence: "This movie was surprisingly good!"
    
    
    Prompt: 
    What is the sentiment of the following sentence? 
    "This movie was surprisingly good!"
    
    Output:
    Positive
    

In this case, the prompt is straightforward, directly asking for the sentiment without providing any prior examples. The LLM can successfully infer that the sentence conveys a positive sentiment based on its understanding of the words "surprisingly" and "good" within the given context.

### 2.3 The Role of Instruction Tuning and RLHF in Zero-Shot Learning

The efficacy of zero-shot prompting is significantly enhanced by techniques like instruction tuning and Reinforcement Learning from Human Feedback (RLHF). Instruction tuning involves further training LLMs on datasets structured as instructions and responses. This fine-tuning process enables models to better align with human expectations regarding instruction following and response generation.

RLHF takes this a step further by incorporating human feedback to optimize the model's performance. Human evaluators assess the quality and relevance of the LLM's responses, providing feedback that is used to refine the model's behavior over time. This iterative training process makes LLMs more adept at interpreting natural language instructions and generating responses that align with human preferences.

## 3\. Limitations of Zero-Shot Prompting and When to Consider Alternatives

### 3.1 Scenarios Where Zero-Shot Prompting Might Fall Short

While remarkably powerful, zero-shot prompting is not without its limitations. Its reliance on implicit understanding can sometimes lead to inconsistencies in performance, particularly when dealing with:

  * **Complex or Nuanced Tasks:** Tasks that demand deep contextual understanding, reasoning, or domain-specific knowledge might pose challenges for zero-shot prompting. The lack of explicit examples could hinder the model's ability to grasp intricate relationships or subtleties within the task.
  * **Ambiguous or Incomplete Instructions:** Clear and unambiguous instructions are crucial for zero-shot prompting. If the prompt is vague or lacks essential context, the LLM may struggle to infer the intended meaning or generate accurate responses.
  * **Data Bias Amplification:** LLMs are trained on massive datasets, which may inadvertently contain biases present in the data. Zero-shot prompting could potentially amplify these biases, leading to unfair or skewed outcomes, especially in tasks involving sensitive attributes or demographic information.



### 3.2 Transitioning to Few-Shot Prompting for Enhanced Performance

In situations where zero-shot prompting falls short, few-shot prompting offers a valuable alternative. This approach involves providing the LLM with a limited number of examples within the prompt itself, guiding the model towards a better understanding of the task and desired output.

Few-shot prompting can be particularly beneficial when:

  * **Task Specificity is High:** For highly specialized tasks or domains, a few well-chosen examples can significantly improve the LLM's performance by providing concrete illustrations of input-output mappings.
  * **Context is Crucial:** In tasks heavily reliant on context, demonstrating the desired output for similar scenarios within the prompt can help the LLM better grasp the nuances and generate more accurate responses.
  * **Mitigating Bias:** While not a complete solution, providing diverse and balanced examples during few-shot prompting can help mitigate bias by exposing the LLM to a wider range of perspectives and reducing the influence of potentially biased pretraining data.



By strategically transitioning to few-shot prompting when necessary, developers and NLP practitioners can overcome some limitations of zero-shot prompting and unlock the full potential of LLMs for a broader set of tasks and domains.

## 4\. Best Practices for Crafting Effective Zero-Shot Prompts

### 4.1 Using Clear and Concise Language

Clarity is paramount when designing zero-shot prompts. Avoid using jargon, overly technical terms, or complex sentence structures that might confuse the LLM. Opt for straightforward language that accurately conveys the desired task and leaves no room for misinterpretation.

### 4.2 Providing Context and Specificity

While zero-shot prompting relies on implicit understanding, providing sufficient context can significantly improve the accuracy and relevance of the LLM's responses. Clearly define the task's purpose, specify any relevant constraints, and offer background information that aids the model in grasping the nuances of the request.

For instance, instead of simply asking, "Translate this sentence into Spanish," provide the LLM with context: "You are a language tutor helping a student learn Spanish. Translate the following sentence into Spanish." This additional information helps the model adopt a specific persona and tailor its response accordingly.

### 4.3 Experimenting with Different Prompt Structures

There is no one-size-fits-all approach to constructing zero-shot prompts. Experimenting with different structures, such as question-answering formats, imperative commands, or role-playing scenarios, can yield valuable insights into what elicits the best performance from the LLM for a particular task.

Iteratively refining and testing different prompt variations, while analyzing the LLM's responses, allows you to identify the most effective strategies for eliciting desired outcomes and achieving optimal results in a zero-shot setting.

## 5\. Applications of Zero-Shot Prompting

Zero-shot prompting, with its ability to elicit remarkable capabilities from large language models, finds applications across a myriad of NLP tasks, transforming the landscape of how we interact with and leverage these powerful AI tools.

### 5.1 Text Classification

Categorizing text into predefined labels is a cornerstone of many NLP applications. Zero-shot prompting enables text classification without the need for task-specific training data. By simply instructing the LLM on the desired categories and providing the text, one can obtain accurate classifications for sentiment analysis, topic detection, spam filtering, and more.

### 5.2 Question Answering

Leveraging the vast knowledge encoded within LLMs, zero-shot prompting facilitates open-domain question answering. Given a question, the LLM can process the query, retrieve relevant information from its internal knowledge stores, and generate comprehensive answers, effectively acting as a powerful search engine.

### 5.3 Summarization

Condensing large volumes of text into concise summaries is another area where zero-shot prompting excels. By simply prompting an LLM with "Summarize the following text," one can obtain accurate and coherent summaries of articles, documents, or conversations, enabling efficient information extraction and knowledge distillation.

## 6\. Conclusion

### 6.1 Summary of Zero-Shot Prompting and Its Significance

Zero-shot prompting shows of the evolution of large language models, showcasing their ability to generalize knowledge and perform tasks without explicit training. This paradigm shift in NLP unlocks unprecedented efficiency, flexibility, and scalability in AI applications, opening doors to a future where interactions with machines become increasingly intuitive and human-like.

### 6.2 Looking Ahead: Advancements and Future Directions

As LLMs continue to evolve in complexity and capability, we can anticipate even more innovative applications of zero-shot prompting, pushing the boundaries of what's possible in artificial intelligence. Research into enhancing prompt engineering techniques, addressing potential biases, and developing more robust evaluation metrics will be crucial for unlocking the full potential of zero-shot prompting and shaping the future of human-machine collaboration.

## FAQs

**What is the difference between zero-shot and few-shot prompting?**

Zero-shot prompting tasks the LLM with performing a task without any prior examples, relying solely on its internalized knowledge from pretraining. Few-shot prompting, on the other hand, involves providing the model with a small number of examples within the prompt to illustrate the desired task and output format.

**Can zero-shot prompting be used for any NLP task?**

While remarkably versatile, zero-shot prompting might not be suitable for all NLP tasks. It excels in tasks where the LLM can leverage its general knowledge and language understanding. For highly specialized tasks or those requiring deep contextual understanding, few-shot prompting or other fine-tuning methods might be more appropriate.

**What are the limitations of zero-shot prompting?**

Zero-shot prompting can face challenges with complex or nuanced tasks that demand intricate reasoning or domain expertise. Additionally, ambiguous instructions and the potential for amplifying biases present in the training data are limitations to consider.

**How can I improve the performance of my zero-shot prompts?**

Crafting clear, concise, and context-rich prompts is crucial for effective zero-shot prompting. Experimenting with different prompt structures, providing specific instructions, and iteratively refining prompts based on the LLM's responses can enhance performance.

Share [ _ _ Tweet ](https://twitter.com/share?text=Zero-Shot%20Prompting%3A%20A%20Powerful%20Technique%20for%20LLMs&url=https://promptengineering.org/zero-shot-prompting-a-powerful-technique-for-llms/ "Share on Twitter") [ _ _ Share ](https://www.facebook.com/sharer.php?u=https://promptengineering.org/zero-shot-prompting-a-powerful-technique-for-llms/ "Share on Facebook") [ _ _ Share ](https://www.linkedin.com/shareArticle?mini=true&url=https://promptengineering.org/zero-shot-prompting-a-powerful-technique-for-llms/&title=Zero-Shot%20Prompting%3A%20A%20Powerful%20Technique%20for%20LLMs&summary=Zero-Shot%20Prompting%3A%20A%20Powerful%20Technique%20for%20LLMs "Share on Linkedin") [ _ _ Email ](/cdn-cgi/l/email-protection#cdf2beb8afa7a8aeb9f097a8bfa2e09ea5a2b9e8fffd9dbfa2a0bdb9a4a3aae8fe8ce8fffd8ce8fffd9da2baa8bfabb8a1e8fffd99a8aea5a3a4bcb8a8e8fffdaba2bfe8fffd818180beebafa2a9b4f0a5b9b9bdbef7e2e2bdbfa2a0bdb9a8a3aaa4a3a8a8bfa4a3aae3a2bfaae2b7a8bfa2e0bea5a2b9e0bdbfa2a0bdb9a4a3aae0ace0bda2baa8bfabb8a1e0b9a8aea5a3a4bcb8a8e0aba2bfe0a1a1a0bee2eba3afbebdf697a8bfa2e09ea5a2b9e8fffd9dbfa2a0bdb9a4a3aae8fe8ce8fffd8ce8fffd9da2baa8bfabb8a1e8fffd99a8aea5a3a4bcb8a8e8fffdaba2bfe8fffd818180be "Share by email") _ _ Copy

## Read next

  * [LLM](/tag/llm/)
  * [Discussion](/tag/discussion/)
  * [Lesson](/tag/lesson/)

Feb 13, 2025

###  [The Evolution of Language AI & LLMs](/the-evolution-of-language-ai-l/)

Language AI has evolved from simple word counting to sophisticated models like transformers, aiming to preserve meaning through numerical representation, with future breakthroughs poised to enhance reasoning and contextual understanding. 

[ _ _ ](/the-evolution-of-language-ai-l/)

  * [Lesson](/tag/lesson/)
  * [LLM](/tag/llm/)
  * [Evaluation](/tag/evaluation/)

Feb 8, 2025

###  [Beyond the Hype - How to Test LLM for Intelligence, Accuracy, and Reliability](/beyond-the-hype-how-to-test-llm-for-intelligence-accuracy-and-reliability/)

The LLM T.E.S.T. Framework is a structured approach for evaluating Large Language Models (LLMs) across multiple dimensions. It determines an AI's true capabilities, reliability, and scalability for real-world applications, distinguishing truly useful models from those that merely appear intelligent. 

[ _ _ ](/beyond-the-hype-how-to-test-llm-for-intelligence-accuracy-and-reliability/)

  * [Lesson](/tag/lesson/)
  * [RAG](/tag/rag/)
  * [Prompt Engineering](/tag/prompt-engineering/)

Nov 4, 2024

###  [Optimizing Small-Scale RAG Systems: Techniques for Efficient Data Retrieval and Enhanced Performance](/optimizing-small-scale-rag-systems-techniques-for-efficient-data-retrieval-and-enhanced-performance/)

Learn key techniques to optimize small-scale RAG systems for efficient, accurate data retrieval and enhanced performance. 

[ _ _ ](/optimizing-small-scale-rag-systems-techniques-for-efficient-data-retrieval-and-enhanced-performance/)

[ ![Prompt Engineering Institute ](https://digitalpress.fra1.cdn.digitaloceanspaces.com/75xl145/2024/03/prompt-engineering-and-ai-institute-2.png) ](https://promptengineering.org)

Hands-on Training, Expert-led Courses, and Real-World AI Applications with Certifications and Support

Subscribe

Great! Check your inbox and click the link.

Sorry, something went wrong. Please try again.

Links

  * [ Learn ](https://promptengineering.org/Learn/)
  * [ Free Course ](https://promptengineering.org/prompt-engineering-course/)
  * [ About ](https://promptengineering.org/about/)
  * [ Contact ](https://promptengineering.org/contact-us/)
  * [ Community ](https://promptengineer.net/communities/groups/prompt-engineering-community/private-group)



  * Sign up
  * [ Contact Us ](https://promptengineering.org/contact-us/)



Socials [_ _Twitter](https://twitter.com/PromptOrg) _ _RSS _ _Threads _ _Linkedin _ _Discord _ Download more icon variants from https://tabler-icons.io/i/brand-github _Github _ Download more icon variants from https://tabler-icons.io/i/brand-instagram _Instagram _ _Pinterest _ _Reddit _ Download more icon variants from https://tabler-icons.io/i/brand-telegram _Telegram _ Download more icon variants from https://tabler-icons.io/i/brand-tiktok _Tiktok _ Download more icon variants from https://tabler-icons.io/i/brand-whatsapp _Whatsapp _ Download more icon variants from https://tabler-icons.io/i/brand-youtube _Youtube

(C)2025 [Prompt Engineering Institute ](https://promptengineering.org). Published with [Ghost](https://ghost.org) & [Array](https://brightthemes.com/themes/array).

_ _


url: https://www.leewayhertz.com/prompt-engineering/
title: Prompt engineering: The process, uses, techniques, applications and best practices
scrape_date: 2025-03-17T12:18:26.915546
content_type: markdown
hash: 4dc5a5c2c6f906c72a40590b968935642c01f199b8559307676fb5346034a41c
content:
The Hackett Group Announces Strategic Acquisition of Leading Gen AI Development Firm LeewayHertz

[Read more](https://www.thehackettgroup.com/the-hackett-group-announces-strategic-acquisition-of-leading-gen-ai-development-firm-leewayhertz/)

[ ![LeewayHertz - AI Development Company](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/09/16071313/LeewayHertz-A-Hackett-Group-Company.svg) ](https://www.leewayhertz.com/)

ToggleToggle

  * [AI PRODUCTS](https://www.leewayhertz.com/enterprise-generative-ai-platform/)
    *       *         * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/16011703/Group-2943.webp)](https://www.leewayhertz.com/enterprise-generative-ai-platform/)
        * **[Enterprise GenAI Platform](https://www.leewayhertz.com/enterprise-generative-ai-platform/)**

      *         * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/12023543/Group-3012.svg)](https://www.leewayhertz.com/ai-copilot-for-sales/)
        * **[AI Copilot for Sales](https://www.leewayhertz.com/ai-copilot-for-sales/)**

      *         * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/12023648/Group-3010.svg)](https://www.leewayhertz.com/ai-research-solution-for-due-diligence/)
        * **[AI Research Solution for Due Diligence](https://www.leewayhertz.com/ai-research-solution-for-due-diligence/)**

      *         * [![ChatGPT for Healthcare](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![ChatGPT for Healthcare](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/05/18225843/ChatGPT-for-Healthcare-menu.svg)](https://www.leewayhertz.com/generative-ai-platform-for-healthcare/)
        * **[GenAI Platform for Healthcare](https://www.leewayhertz.com/generative-ai-platform-for-healthcare/)**

    *       *         * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/05/16235357/Group-3014.svg)](https://www.leewayhertz.com/ai-customer-service-agent/)
        * **[AI Customer Service Agent](https://www.leewayhertz.com/ai-customer-service-agent/)**

      *         * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/05/18224152/ChatGPT-for-Finance-menu.svg)](https://www.leewayhertz.com/generative-ai-platform-for-finance/)
        * **[GenAI Platform for Finance](https://www.leewayhertz.com/generative-ai-platform-for-finance/)**

      *         * [![ChatGPT for Manufacturing](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![ChatGPT for Manufacturing](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/05/23050527/ChatGPT-for-Manufacturing-menu.svg)](https://www.leewayhertz.com/generative-ai-platform-for-manufacturing/)
        * **[GenAI Platform for Manufacturing](https://www.leewayhertz.com/generative-ai-platform-for-manufacturing/)**

      *         * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/05/19020738/ChatGPT-for-Logistics-menu.svg)](https://www.leewayhertz.com/generative-ai-platform-for-logistics/)
        * **[GenAI Platform for Logistics](https://www.leewayhertz.com/generative-ai-platform-for-logistics/)**

  * [SERVICES](https://www.leewayhertz.com/services/)
    * [Generative AI](https://www.leewayhertz.com/generative-ai-development-company/?utm_source=menu)
      *         *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/10071939/Generative-AI-Development-menuicon.svg)](https://www.leewayhertz.com/generative-ai-development-company/?utm_source=menu)
          * [**Generative AI Development**](https://www.leewayhertz.com/generative-ai-development-company/?utm_source=menu)

        *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/06/26054947/Group-2964.svg)](https://www.leewayhertz.com/generative-ai-integration-service/?utm_source=menu)
          * [**Generative AI Integration Services**](https://www.leewayhertz.com/generative-ai-integration-service/?utm_source=menu)

        *           * [![Generative AI Consulting Company menu](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/07/05035126/Generative-AI-Consulting-Company-menu.svg)](https://www.leewayhertz.com/generative-ai-consulting-company/?utm_source=menu)
          * [**Generative AI Consulting Company**](https://www.leewayhertz.com/generative-ai-consulting-company/?utm_source=menu)

        *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/09/04022309/Hire-Generative-AI-Engineers-menu.svg)](https://www.leewayhertz.com/hire-generative-ai-engineers/?utm_source=menu)
          * [**Hire Generative AI Engineers**](https://www.leewayhertz.com/hire-generative-ai-engineers/?utm_source=menu)

      *         *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/02/25221821/AI-Agent-Development-company-menuicon.svg)](https://www.leewayhertz.com/ai-agent-development-company/?utm_source=menu)
          * [**AI Agent Development**](https://www.leewayhertz.com/ai-agent-development-company/?utm_source=menu)

        *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/02/25222330/AI-Copilot-Development-Company-menuimage.svg)](https://www.leewayhertz.com/ai-copilot-development-company/?utm_source=menu)
          * [**AI Copilot Development**](https://www.leewayhertz.com/ai-copilot-development-company/?utm_source=menu)

        *           * [![AI Marketing Agent Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/07/26081055/AI-Marketing-Agent-Development.svg)](https://www.leewayhertz.com/ai-marketing-agent-development/?utm_source=menu)
          * [**AI Marketing Agent Development**](https://www.leewayhertz.com/ai-marketing-agent-development/?utm_source=menu)

        *           * [![Hire Prompt Engineers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Hire Prompt Engineers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/10072357/Hire-Prompt-Engineers-menuicon.svg)](https://www.leewayhertz.com/hire-prompt-engineers/?utm_source=menu)
          * [**Hire Prompt Engineers**](https://www.leewayhertz.com/hire-prompt-engineers/?utm_source=menu)

      *         *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/10072243/Adaptive-AI-Development-Company-menuicon.svg)](https://www.leewayhertz.com/adaptive-ai-development-company/?utm_source=menu)
          * [**Adaptive AI Development Company**](https://www.leewayhertz.com/adaptive-ai-development-company/?utm_source=menu)

        *           * [![ChatGPT Developers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![ChatGPT Developers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/13221413/ChatGPT-Developers-Menu.svg)](https://www.leewayhertz.com/chatgpt-developers/?utm_source=menu)
          * [**ChatGPT Developers**](https://www.leewayhertz.com/chatgpt-developers/?utm_source=menu)

        *           * [![Stable diffusion developers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Stable diffusion developers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/12/21223447/Stable-diffusion-developers-menu-image.png)](https://www.leewayhertz.com/stable-diffusion-developers/?utm_source=menu)
          * [**Stable Diffusion Developers**](https://www.leewayhertz.com/stable-diffusion-developers/?utm_source=menu)

        *           * [![Large Language Model Development Company](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/11222521/Large-Language-Model-Development-Company_menu.svg)](https://www.leewayhertz.com/large-language-model-development-company/?utm_source=menu)
          * [**Large Language Model Development**](https://www.leewayhertz.com/large-language-model-development-company/?utm_source=menu)

    * [Artificial Intelligence & ML](https://www.leewayhertz.com/ai-development-services-company/?utm_source=menu)
      *         *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/10072828/AI-Development_icon.svg)](https://www.leewayhertz.com/ai-development-services-company/?utm_source=menu)
          * [**AI Development**](https://www.leewayhertz.com/ai-development-services-company/?utm_source=menu)

        *           * [![AI Consulting](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![AI Consulting](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/25220314/AI-Consulting-Menu.svg)](https://www.leewayhertz.com/ai-consulting-services-company/?utm_source=menu)
          * [**AI Consulting**](https://www.leewayhertz.com/ai-consulting-services-company/?utm_source=menu)

        *           * [![AI Deployment Services Company](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/09/10045207/ai-dev-01.svg)](https://www.leewayhertz.com/ai-deployment-services-company/?utm_source=menu)
          * [**AI Deployment Services**](https://www.leewayhertz.com/ai-deployment-services-company/?utm_source=menu)

        *           * [![AI Engineers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![AI Engineers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/10072928/AI-Engineers_icon.svg)](https://www.leewayhertz.com/hire-ai-engineers/?utm_source=menu)
          * [**Hire AI Engineers**](https://www.leewayhertz.com/hire-ai-engineers/?utm_source=menu)

      *         *           * [![AI as a service](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![AI as a service](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/07/21013022/AI-as-a-service.svg)](https://www.leewayhertz.com/ai-as-a-service/?utm_source=menu)
          * **[AI as a Service](https://www.leewayhertz.com/ai-as-a-service/?utm_source=menu)**

        *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/10073152/Hire-Action-Transformer-Developers_icon.svg)](https://www.leewayhertz.com/hire-action-transformer-developers/?utm_source=menu)
          * [**Hire Action Transformer Developers**](https://www.leewayhertz.com/hire-action-transformer-developers/?utm_source=menu)

        *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/30215833/Enterprise-AI-Development-Company-Menu.svg)](https://www.leewayhertz.com/enterprise-ai-development-company/?utm_source=menu)
          * [**Enterprise AI Development**](https://www.leewayhertz.com/enterprise-ai-development-company/?utm_source=menu)

        *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/06/15013042/Group-2947.svg)](https://www.leewayhertz.com/ai-chatbot-development-company/?utm_source=menu)
          * [**AI Chatbot Development Company**](https://www.leewayhertz.com/ai-chatbot-development-company/?utm_source=menu)

      *         *           * [![Enterprise AI Chatbot Development Company](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Enterprise AI Chatbot Development Company](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/07/27010942/Enterprise-AI-Chatbot-Development-Company.svg)](https://www.leewayhertz.com/enterprise-ai-chatbot-development/?utm_source=menu)
          * [**Enterprise AI Chatbot Development Company**](https://www.leewayhertz.com/enterprise-ai-chatbot-development/?utm_source=menu)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/10214015/Transformer-Model-development-menuicon.svg)](https://www.leewayhertz.com/transformer-model-development-services/?utm_source=menu)
          * [**Transformer Model Development**](https://www.leewayhertz.com/transformer-model-development-services/?utm_source=menu)

        *           * [![Machine Learning Operations Menu](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Machine Learning Operations Menu](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/13234945/Machine-Learning-Operations-Menu.svg)](https://www.leewayhertz.com/mlops-consulting-services/?utm_source=menu)
          * [**MLOps Consulting Services**](https://www.leewayhertz.com/mlops-consulting-services/?utm_source=menu)

        *     * [Data Engineering](https://www.leewayhertz.com/data-engineering-services/?utm_source=menu)
      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/03/06020346/Data-Engineering-Services-menu.png)](https://www.leewayhertz.com/data-engineering-services/?utm_source=menu)
          * [**Data Engineering Services**](https://www.leewayhertz.com/data-engineering-services/?utm_source=menu)

        *           * [![Hire data scientists](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Hire data scientists](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/19224144/Hire-data-scientists-menu.svg)](https://www.leewayhertz.com/hire-data-scientists/?utm_source=menu)
          * [**Hire Data Scientist**](https://www.leewayhertz.com/hire-data-scientists/?utm_source=menu)

        *           * [![Data Analytics Service](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Data Analytics Service](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/21065718/Data-Analytics-Service-menu.svg)](https://www.leewayhertz.com/data-analytics-services/?utm_source=menu)
          * [**Data Analytics Services**](https://www.leewayhertz.com/data-analytics-services/?utm_source=menu)

        *           * [![Data Annotation Services](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Data Annotation Services](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/24042949/Data-Annotation-Services-Menu.svg)](https://www.leewayhertz.com/data-annotation-services/?utm_source=menu)
          * [**Data Annotation Services**](https://www.leewayhertz.com/data-annotation-services/?utm_source=menu)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/04/10232340/ML-model_menuicon.svg)](https://www.leewayhertz.com/ml-model-engineering-services/?utm_source=menu)
          * [**ML Model Engineering**](https://www.leewayhertz.com/ml-model-engineering-services/?utm_source=menu)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/06/22224450/Machine-Learning-Development-Icon.svg)](https://www.leewayhertz.com/machine-learning-development-services/?utm_source=menu)
          * [**Machine Learning Development**](https://www.leewayhertz.com/machine-learning-development-services/?utm_source=menu)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/09/18031456/Machine-Learning-and-Data-Science-Consulting-Services-menu.svg)](https://www.leewayhertz.com/machine-learning-and-data-science-consulting-services/?utm_source=menu)
          * [**ML and Data Science Consulting**](https://www.leewayhertz.com/machine-learning-and-data-science-consulting-services/?utm_source=menu)

        *           * [![Big Data Consulting](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Big Data Consulting](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/07/11233337/Big-Data-Consulting-menu.svg)](https://www.leewayhertz.com/big-data-consulting-company/?utm_source=menu)
          * [**Big Data Consulting**](https://www.leewayhertz.com/big-data-consulting-company/?utm_source=menu)

    * [Web3](https://www.leewayhertz.com/web3-development-company/?utm_source=menu)
      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/07/25031645/Web3-development_menu.svg)](https://www.leewayhertz.com/web3-development-company/?utm_source=menu)
          * [**Web3 Development**](https://www.leewayhertz.com/web3-development-company/?utm_source=menu)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/07/25031644/State-of-web3_menu.svg)](https://www.leewayhertz.com/current-state-of-web3/?utm_source=menu)
          * [**State of Web3**](https://www.leewayhertz.com/current-state-of-web3/?utm_source=menu)

        *           * [![Rust Development](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Rust Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/12/21224015/Rust-Development-menu.png)](https://www.leewayhertz.com/rust-development-company/?utm_source=menu)
          * [**Rust Development**](https://www.leewayhertz.com/rust-development-company/?utm_source=menu)

        *           * [![Web3 Game Development](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Web3 Game Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/12/06010250/Web3-Game-Development.svg)](https://www.leewayhertz.com/web3-game-development-company/?utm_source=menu)
          * [**Web3 Game Development**](https://www.leewayhertz.com/web3-game-development-company/?utm_source=menu)

      *         *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/07/21010147/metaverse_development_company_menu.svg)](https://www.leewayhertz.com/metaverse-development-company/?utm_source=menu)
          * [**Metaverse Development**](https://www.leewayhertz.com/metaverse-development-company/?utm_source=menu)

        *           * [![NFT Marketplace Development](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![NFT Marketplace Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/20000042/NFT-Marketplace-Development.svg)](https://www.leewayhertz.com/nft-marketplace-development-company/?utm_source=menu)
          * **[NFT Marketplace Development](https://www.leewayhertz.com/nft-marketplace-development-company/?utm_source=menu)**

        *         *     * [Blockchain](https://www.leewayhertz.com/blockchain-consulting-services/?utm_source=menu)
      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20212%20131'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/08/22234107/blockchain_menu_img.jpg)](https://www.leewayhertz.com/blockchain-development-company/?utm_source=menu)
          * [**Blockchain Development**](https://www.leewayhertz.com/blockchain-development-company/?utm_source=menu)

        *           * [![Blockchain Consulting Services](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Blockchain Consulting Services](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/15200115/Blockchain-Consulting-Services-menu.svg)](https://www.leewayhertz.com/blockchain-consulting-services/?utm_source=menu)
          * **[Blockchain Consulting](https://www.leewayhertz.com/blockchain-consulting-services/?utm_source=menu)**

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20212%20131'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/08/22231718/substrate_menu_img%402x.jpg)](https://www.leewayhertz.com/substrate-development-services/?utm_source=menu)
          * **[Substrate Development](https://www.leewayhertz.com/substrate-development-services/?utm_source=menu)**

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/15051309/Ethereum-Blockchain-Services-menu.webp)](https://www.leewayhertz.com/ethereum-dapp-development-company/?utm_source=menu)
          * [**Ethereum dApp Development Company**](https://www.leewayhertz.com/ethereum-dapp-development-company/?utm_source=menu)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20212%20131'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/08/22232445/hyperledger_menu_img-300x186-1.jpg)](https://www.leewayhertz.com/hyperledger-blockchain-development-services/?utm_source=menu)
          * **[Hyperledger Development](https://www.leewayhertz.com/hyperledger-blockchain-development-services/?utm_source=menu)**

        *           * [![Hire Golang Developers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Hire Golang Developers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/12/21224800/Hire-Golang-Developers-menu-image.png)](https://www.leewayhertz.com/golang-development-company/?utm_source=menu)
          * **[Golang Development](https://www.leewayhertz.com/golang-development-company/?utm_source=menu)**

        *           * [![Cosmos Blockchain Development company menu](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Cosmos Blockchain Development company menu](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/12/21225700/Cosmos-Blockchain-Development-company-menu.png)](https://www.leewayhertz.com/cosmos-development-company/?utm_source=menu)
          * **[Cosmos Development](https://www.leewayhertz.com/cosmos-development-company/?utm_source=menu)**

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20212%20131'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/08/22234836/solana_menu_img-300x186-1.jpg)](https://www.leewayhertz.com/solana-blockchain-development-company/?utm_source=menu)
          * **[Solana Development](https://www.leewayhertz.com/solana-blockchain-development-company/?utm_source=menu)**

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20212%20131'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/08/22232945/tezos_menu_img.jpg)](https://www.leewayhertz.com/tezos-development/?utm_source=menu)
          * **[Tezos Development](https://www.leewayhertz.com/tezos-development/?utm_source=menu)**

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20212%20131'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/08/22233827/stellar_menu_img.jpg)](https://www.leewayhertz.com/stellar-blockchain/?utm_source=menu)
          * **[Stellar Development](https://www.leewayhertz.com/stellar-blockchain/?utm_source=menu)**

        *           * [![Smart Contract Audit](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Smart Contract Audit](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/11/16023317/Smart-Contract-Audit.svg)](https://www.leewayhertz.com/smart-contract-audit-company/?utm_source=menu)
          * **[Smart Contract Audit](https://www.leewayhertz.com/smart-contract-audit-company/?utm_source=menu)**

        *           * [![crypto wallet](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![crypto wallet](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/12/27110530/crypto-wallet.svg)](https://www.leewayhertz.com/crypto-wallet-development-solution/?utm_source=menu)
          * **[Crypto Wallet Development Solution](https://www.leewayhertz.com/crypto-wallet-development-solution/?utm_source=menu)**

    * [Software Development](https://www.leewayhertz.com/software-development-company/?utm_source=menu)
      *         *           * [![Software Development](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Software Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030720/Group-81.svg)](https://www.leewayhertz.com/software-development-company/?utm_source=menu)
          * [**Software Development**](https://www.leewayhertz.com/software-development-company/?utm_source=menu)

        *           * [![SAAS Development](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![SAAS Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030721/Group-82.svg)](https://www.leewayhertz.com/saas-development-company/?utm_source=menu)
          * **[SaaS Development](https://www.leewayhertz.com/saas-development-company/?utm_source=menu)**

        *           * [![Software Consulting](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Software Consulting](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13042859/Group-83-1.svg)](https://www.leewayhertz.com/software-consulting-company/?utm_source=menu)
          * [**Software Consulting**](https://www.leewayhertz.com/software-consulting-company/?utm_source=menu)

        *           * [![UI UX](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![UI UX](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030719/Group-84.svg)](https://www.leewayhertz.com/ui-ux-design-services/?utm_source=menu)
          * [**UI/UX Design Service**](https://www.leewayhertz.com/ui-ux-design-services/?utm_source=menu)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030717/Group-99.svg)](https://www.leewayhertz.com/enterprise-software-development-company/?utm_source=menu)
          * [**Enterprise Software Development**](https://www.leewayhertz.com/enterprise-software-development-company/?utm_source=menu)

        *           * [![Web App Development](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Web App Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13043058/Group-88-1.svg)](https://www.leewayhertz.com/web-application-development/?utm_source=menu)
          * **[Web Application Development](https://www.leewayhertz.com/web-application-development/?utm_source=menu)**

        *           * [![Digital Transformation](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Digital Transformation](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13043214/Group-87-1.svg)](https://www.leewayhertz.com/digital-transformation-services/?utm_source=menu)
          * [**Digital Transformation**](https://www.leewayhertz.com/digital-transformation-services/?utm_source=menu)

        *     * [Hire Developers](https://www.leewayhertz.com/hire-dedicated-developers/?utm_source=menu)
      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/09/28012619/Hire-Machine-Learning-Developers-menu.svg)](https://www.leewayhertz.com/hire-machine-learning-developers/?utm_source=menu)
          * [**Hire ML Developers**](https://www.leewayhertz.com/hire-machine-learning-developers/?utm_source=menu)

        *           * [![DevOps Engineers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![DevOps Engineers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030727/Group-91.svg)](https://www.leewayhertz.com/hire-devops-engineer/?utm_source=menu)
          * [**DevOps Engineers**](https://www.leewayhertz.com/hire-devops-engineer/?utm_source=menu)

        *           * [![Offshore Engineers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Offshore Engineers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13040611/Group-92-1.svg)](https://www.leewayhertz.com/hire-offshore-engineer/?utm_source=menu)
          * [**Offshore Engineers**](https://www.leewayhertz.com/hire-offshore-engineer/?utm_source=menu)

        *           * [![App Developer](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![App Developer](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030717/Group-99.svg)](https://www.leewayhertz.com/app-developer-united-states/?utm_source=menu)
          * [**App Developer**](https://www.leewayhertz.com/app-developer-united-states/?utm_source=menu)

      *         *           * [![Hire Golang Developers menu](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Hire Golang Developers menu](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/12/21225849/Hire-Golang-Developers-menu.png)](https://www.leewayhertz.com/hire-golang-developers/?utm_source=menu)
          * [**Hire Golang Developers**](https://www.leewayhertz.com/hire-golang-developers/?utm_source=menu)

        *           * [![Blockchain Engineers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Blockchain Engineers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13040737/Group-93-1.svg)](https://www.leewayhertz.com/blockchain-developers-uae/?utm_source=menu)
          * [**Blockchain Developer**](https://www.leewayhertz.com/blockchain-developers-uae/?utm_source=menu)

        *           * [![Hire Dedicated Developers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Hire Dedicated Developers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13041414/Group-96-1.svg)](https://www.leewayhertz.com/hire-dedicated-developers/?utm_source=menu)
          * [**Hire Dedicated Developers**](https://www.leewayhertz.com/hire-dedicated-developers/?utm_source=menu)

        *           * [![Hire Cosmos Developers menu](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Hire Cosmos Developers menu](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/12/21230104/Hire-Cosmos-Developers-menu.png)](https://www.leewayhertz.com/hire-cosmos-developers/?utm_source=menu)
          * [**Hire Cosmos Developers**](https://www.leewayhertz.com/hire-cosmos-developers/?utm_source=menu)

      *         *           * [![Hire Stellar Developers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Hire Stellar Developers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030726/Group-94.svg)](https://www.leewayhertz.com/hire-stellar-developers/?utm_source=menu)
          * [**Hire Stellar Developers**](https://www.leewayhertz.com/hire-stellar-developers/?utm_source=menu)

        *           * [![Full Stack Developers](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Full Stack Developers](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13040954/Group-97-1.svg)](https://www.leewayhertz.com/hire-full-stack-developer/?utm_source=menu)
          * [**Full Stack Developer**](https://www.leewayhertz.com/hire-full-stack-developer/?utm_source=menu)

        *         *     * [Internet of Things (IoT)](https://www.leewayhertz.com/iot-development-company/?utm_source=menu)
      *         *           * [![IoT Development](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![IoT Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030724/Group-28.svg)](https://www.leewayhertz.com/iot-development-company/?utm_source=menu)
          * [**IoT Development**](https://www.leewayhertz.com/iot-development-company/?utm_source=menu)

        *           * [![IoT Solutions](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![IoT Solutions](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030722/Group-29.svg)](https://www.leewayhertz.com/industrial-iot-solutions/?utm_source=menu)
          * **[Industrial IoT Solutions](https://www.leewayhertz.com/industrial-iot-solutions/?utm_source=menu)**

        *           * [![Firmware Development](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Firmware Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030724/Group-30.svg)](https://www.leewayhertz.com/firmware-development-company/?utm_source=menu)
          * **[Firmware Development](https://www.leewayhertz.com/firmware-development-company/?utm_source=menu)**

        *           * [![IoT Healthcare](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![IoT Healthcare](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030725/Group-32.svg)](https://www.leewayhertz.com/iot-healthcare-software-development/?utm_source=menu)
          * [**IoT Healthcare Software**](https://www.leewayhertz.com/iot-healthcare-software-development/?utm_source=menu)

      *         *           * [![IoT Product](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![IoT Product](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030721/Group-31-2.svg)](https://www.leewayhertz.com/iot-product-development/?utm_source=menu)
          * **[IoT Product Development](https://www.leewayhertz.com/iot-product-development/?utm_source=menu)**

        *           * [![Hardware Design](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Hardware Design](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030723/Group-33-3.svg)](https://www.leewayhertz.com/iot-hardware-design-services/?utm_source=menu)
          * **[Hardware Design](https://www.leewayhertz.com/iot-hardware-design-services/?utm_source=menu)**

        *         *     * [Metaverse](https://www.leewayhertz.com/metaverse/?utm_source=menu)
      *         *           * [![Metaverse Integration](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Metaverse Integration](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13034053/Group-6-1.svg)](https://www.leewayhertz.com/metaverse/integration-services/)
          * [**Metaverse Integration**](https://www.leewayhertz.com/metaverse/integration-services/)

        *           * [![Metaverse 3D Space](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Metaverse 3D Space](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030727/Group-7.svg)](https://www.leewayhertz.com/metaverse/3d-space-development/)
          * [**Metaverse 3D Space**](https://www.leewayhertz.com/metaverse/3d-space-development/)

        *           * [![Metaverse Gaming Space](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Metaverse Gaming Space](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030729/Group-8.svg)](https://www.leewayhertz.com/metaverse/gaming-space-development/)
          * [**Metaverse Gaming Space**](https://www.leewayhertz.com/metaverse/gaming-space-development/)

        *           * [![Metaverse Social Media](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Metaverse Social Media](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030730/Group-9.svg)](https://www.leewayhertz.com/metaverse/social-media-development-services/)
          * [**Metaverse Social Media**](https://www.leewayhertz.com/metaverse/social-media-development-services/)

      *         *           * [![Metaverse Application Development](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Metaverse Application Development](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13034018/Group-11-1.svg)](https://www.leewayhertz.com/metaverse/application-development/)
          * [**Metaverse Application**](https://www.leewayhertz.com/metaverse/application-development/)

        *           * [![Metaverse Decentralized](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Metaverse Decentralized](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/06/13030731/Group-10-2.svg)](https://www.leewayhertz.com/metaverse/decentralized-platform-development/)
          * [**Metaverse Decentralized**](https://www.leewayhertz.com/metaverse/decentralized-platform-development/)

        *           * [![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2022/07/21010147/metaverse_development_company_menu.svg)](https://www.leewayhertz.com/metaverse-development-company/)
          * [**Metaverse Development**](https://www.leewayhertz.com/metaverse-development-company/)

        *     * [Software Consulting](https://www.leewayhertz.com/software-consulting-company/?utm_source=menu)
      *         *           * [![AI_Consulting_company-menu](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20242%20150'%3E%3C/svg%3E)![AI_Consulting_company-menu](https://www.leewayhertz.com/wp-content/uploads/2020/09/AI_Consulting_company-menu.webp)](https://www.leewayhertz.com/ai-consulting-services-company/)
          * [**AI Consulting**](https://www.leewayhertz.com/ai-consulting-services-company/)

        *           * [![RPA Consulting](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20176%20109'%3E%3C/svg%3E)![RPA Consulting](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2021/09/22213250/RPA-Consulting.png)](https://www.leewayhertz.com/rpa-consulting-services/)
          * [**RPA Consulting**](https://www.leewayhertz.com/rpa-consulting-services/)

        *           * [![Blockchain Consulting](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20175%20109'%3E%3C/svg%3E)![Blockchain Consulting](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2021/09/22213233/Blockchain-Consulting-e1681887195285.png)](https://www.leewayhertz.com/blockchain-consulting-services/)
          * [**Blockchain Consulting**](https://www.leewayhertz.com/blockchain-consulting-services/)

        *           * [![DevOps Consulting](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20175%20109'%3E%3C/svg%3E)![DevOps Consulting](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2021/09/22213239/DevOps-Consulting.png)](https://www.leewayhertz.com/devops-consulting/)
          * [**DevOps Consulting**](https://www.leewayhertz.com/devops-consulting/)

      *         *           * [![contactless_softwaredevelopment-menu](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20242%20150'%3E%3C/svg%3E)![contactless_softwaredevelopment-menu](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2020/06/10020604/contactless_softwaredevelopment-menu.jpg)](https://www.leewayhertz.com/covid19-technology-consulting/)
          * [**Covid-19 Technology Consulting**](https://www.leewayhertz.com/covid19-technology-consulting/)

        *           * [![Digital Risk Management](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20175%20109'%3E%3C/svg%3E)![Digital Risk Management](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2021/09/22213240/Digital-Risk-Management-31.png)](https://www.leewayhertz.com/digital-risk-management/)
          * [**Digital Risk Management**](https://www.leewayhertz.com/digital-risk-management/)

        *         *   * [INDUSTRIES](https://www.leewayhertz.com/industries/)
    * [Finance](https://www.leewayhertz.com/fintech-software-development/)
      *         *           * [![fintech-01](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![fintech-01](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/07/15041055/fintech-01.svg)](https://www.leewayhertz.com/fintech-ai-consulting-and-development-company/)
          * [**Fintech AI Consulting and Development Company**](https://www.leewayhertz.com/fintech-ai-consulting-and-development-company/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26053632/Group-43-1.svg)](https://www.leewayhertz.com/fintech-software-development/)
          * [**Fintech Software Development**](https://www.leewayhertz.com/fintech-software-development/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26053800/Group-45.svg)](https://www.leewayhertz.com/ai-use-cases-in-banking-and-finance/)
          * [**AI Use Cases for Banking and Finance**](https://www.leewayhertz.com/ai-use-cases-in-banking-and-finance/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/05/14211035/Group-3013.svg)](https://www.leewayhertz.com/ai-in-due-diligence/)
          * [**AI in Due Diligence**](https://www.leewayhertz.com/ai-in-due-diligence)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26054240/Group-47.svg)](https://www.leewayhertz.com/ai-in-financial-compliance/)
          * [**AI for Financial Compliance**](https://www.leewayhertz.com/ai-in-financial-compliance/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/28234913/Group-76-1.svg)](https://www.leewayhertz.com/generative-ai-in-finance-and-banking/)
          * [**GenAI for Finance and Banking**](https://www.leewayhertz.com/generative-ai-in-finance-and-banking/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/28235216/Group-77.svg)](https://www.leewayhertz.com/generative-ai-in-asset-management/)
          * [**GenAI for Asset Management**](https://www.leewayhertz.com/generative-ai-in-asset-management/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/28235331/Group-50-1.svg)](https://www.leewayhertz.com/ai-loan-underwriting/)
          * [**AI Loan Underwriting**](https://www.leewayhertz.com/ai-loan-underwriting/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/28235705/Group-79.svg)](https://www.leewayhertz.com/ai-based-credit-scoring/)
          * [**AI-based Credit Scoring**](https://www.leewayhertz.com/ai-based-credit-scoring/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29001250/Group-57.svg)](https://www.leewayhertz.com/ai-for-financial-document-processing/)
          * [**AI for Financial Document Processing**](https://www.leewayhertz.com/ai-for-financial-document-processing/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29001139/Group-56.svg)](https://www.leewayhertz.com/ai-in-financial-modeling/)
          * [**AI for Financial Modeling**](https://www.leewayhertz.com/ai-in-financial-modeling/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29000954/Group-74-1.svg)](https://www.leewayhertz.com/ai-in-financial-planning/)
          * [**AI for Financial Planning**](https://www.leewayhertz.com/ai-in-financial-planning/)

    * [Insurance](https://www.leewayhertz.com/insurance-software-development/)
      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/07/02054031/Insurance-AI-Consulting-and-Development-Company-01.svg)](https://www.leewayhertz.com/insurance-ai-consulting-and-development-company/)
          * [**Insurance AI Consulting and Development**](https://www.leewayhertz.com/insurance-ai-consulting-and-development-company/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26055506/Group-62.svg)](https://www.leewayhertz.com/insurance-software-development/)
          * [**Insurance Software Development**](https://www.leewayhertz.com/insurance-software-development/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26055849/Group-81.svg)](https://www.leewayhertz.com/ai-for-insurance/)
          * [**AI for Insurance**](https://www.leewayhertz.com/ai-for-insurance/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26055959/Group-59.svg)](https://www.leewayhertz.com/insurance-claim-processing-using-ai/)
          * [**AI for insurance Claims Processing**](https://www.leewayhertz.com/insurance-claim-processing-using-ai/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26060429/Group-65.svg)](https://www.leewayhertz.com/ai-in-claims-processing/)
          * [**AI for Claims Processing**](https://www.leewayhertz.com/ai-in-claims-processing/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29012737/Group-66-1.svg)](https://www.leewayhertz.com/generative-ai-in-insurance/)
          * [**GenAI for Insurance**](https://www.leewayhertz.com/generative-ai-in-insurance/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29013201/Group-67.svg)](https://www.leewayhertz.com/ai-in-fraud-detection/)
          * [**AI for Fraud Detection**](https://www.leewayhertz.com/ai-in-fraud-detection/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29013327/Group-69.svg)](https://www.leewayhertz.com/build-enterprise-ai-solutions-for-insurance/)
          * [**Build Enterprise AI solutions**](https://www.leewayhertz.com/build-enterprise-ai-solutions-for-insurance/
)

    * [Manufacturing](https://www.leewayhertz.com/manufacturing-software-development/)
      *         *           * [![Manufacturing AI Consulting and Development Company-01](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Manufacturing AI Consulting and Development Company-01](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/07/04043303/Manufacturing-AI-Consulting-and-Development-Company-01.svg)](https://www.leewayhertz.com/manufacturing-ai-consulting-and-development-company/)
          * [**Manufacturing AI Development Company**](https://www.leewayhertz.com/manufacturing-ai-consulting-and-development-company/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26062546/Group-35-1.svg)](https://www.leewayhertz.com/manufacturing-software-development/)
          * [**Manufacturing Software Development**](https://www.leewayhertz.com/manufacturing-software-development/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26063531/Group-44.svg)](https://www.leewayhertz.com/ai-in-demand-forecasting/)
          * [**AI for Demand Forecasting**](https://www.leewayhertz.com/ai-in-demand-forecasting/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26063712/Group-43-2.svg)](https://www.leewayhertz.com/ai-in-anomaly-detection/)
          * [**AI for Anomaly Detection**](https://www.leewayhertz.com/ai-in-anomaly-detection/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29020248/Group-45-1.svg)](https://www.leewayhertz.com/generative-ai-in-manufacturing/)
          * [**GenAI for Manufacturing**](https://www.leewayhertz.com/generative-ai-in-manufacturing/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29020627/Group-38-1.svg)](https://www.leewayhertz.com/ai-in-predictive-maintenance/)
          * [**AI for Predictive Maintenance**](https://www.leewayhertz.com/ai-in-predictive-maintenance/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29020834/Group-42-1.svg)](https://www.leewayhertz.com/ai-in-production-planning/)
          * [**AI for Production Planning**](https://www.leewayhertz.com/ai-in-production-planning/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29021003/Group-41-1.svg)](https://www.leewayhertz.com/ai-in-product-lifecycle-management/)
          * [**AI for Product Lifecycle Management**](https://www.leewayhertz.com/ai-in-product-lifecycle-management/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29021400/Group-39-1.svg)](https://www.leewayhertz.com/build-enterprise-ai-solutions-for-manufacturing/)
          * [**Build Enterprise AI Solutions**](https://www.leewayhertz.com/build-enterprise-ai-solutions-for-manufacturing/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29021542/Group-36.svg)](https://www.leewayhertz.com/how-to-build-predictive-ml-model-for-manufacturing/)
          * [**Build Predictive ML Models**](https://www.leewayhertz.com/how-to-build-predictive-ml-model-for-manufacturing/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29022753/Group-37.svg)](https://www.leewayhertz.com/implement-ai-for-quality-control-and-defect-analysis/)
          * [**AI for Quality Control**](https://www.leewayhertz.com/implement-ai-for-quality-control-and-defect-analysis/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26063412/Group-40.svg)](https://www.leewayhertz.com/ai-use-cases-in-manufacturing/)
          * [**AI Use Cases for Manufacturing**](https://www.leewayhertz.com/ai-use-cases-in-manufacturing/)

    * [Logistics](https://www.leewayhertz.com/logistics-software-development/)
      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/07/08003152/logistics-01.svg)](https://www.leewayhertz.com/logistics-ai-consulting-and-development-company/)
          * [**Logistics AI Consulting and Development Company**](https://www.leewayhertz.com/logistics-ai-consulting-and-development-company/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26060958/Group-34.svg)](https://www.leewayhertz.com/logistics-software-development/)
          * [**Logistics Software Development**](https://www.leewayhertz.com/logistics-software-development/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26061139/Group-25.svg)](https://www.leewayhertz.com/ai-in-logistics-and-supply-chain/)
          * [**AI for Logistics and Supply Chain**](https://www.leewayhertz.com/ai-in-logistics-and-supply-chain/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26061246/Group-30-1.svg)](https://www.leewayhertz.com/ai-in-fleet-management/)
          * [**AI for Fleet Management**](https://www.leewayhertz.com/ai-in-fleet-management/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26061350/Group-29.svg)](https://www.leewayhertz.com/ai-driven-supply-chain-optimization/)
          * [**AI for Supply Chain Optimization**](https://www.leewayhertz.com/ai-driven-supply-chain-optimization/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29023233/Group-31-1.svg)](https://www.leewayhertz.com/automotive-supply-chain-optimization-using-ai/)
          * [**Automotive Supply Chain Optimization**](https://www.leewayhertz.com/automotive-supply-chain-optimization-using-ai/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29023506/Group-28.svg)](https://www.leewayhertz.com/fleet-management-using-ai/)
          * [**Fleet Management Using AI**](https://www.leewayhertz.com/fleet-management-using-ai/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29023614/Group-26.svg)](https://www.leewayhertz.com/efficient-route-optimization-with-ai/)
          * [**Efficient Route Optimization**](https://www.leewayhertz.com/efficient-route-optimization-with-ai/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29023729/Group-27.svg)](https://www.leewayhertz.com/generative-ai-in-supply-chain/)
          * [**GenAI for Supply Chain**](https://www.leewayhertz.com/generative-ai-in-supply-chain/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29024054/Group-24-1.svg)](https://www.leewayhertz.com/build-enterprise-ai-solutions-for-logistics/)
          * [**Build Enterprise AI solutions**](https://www.leewayhertz.com/build-enterprise-ai-solutions-for-logistics/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29024213/Group-23-1.svg)](https://www.leewayhertz.com/ai-for-operational-efficiency/)
          * [**AI for Operational Efficiency**](https://www.leewayhertz.com/ai-for-operational-efficiency/)

        *     * [Hospitality](https://www.leewayhertz.com/ai-use-cases-in-hospitality/)
      *         *           * [![Hospitality AI Consulting and Development company](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Hospitality AI Consulting and Development company](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/08/09001733/hospitatlity-01.svg)](https://www.leewayhertz.com/hospitality-ai-consulting-and-development-company/)
          * [**Hospitality AI Consulting and Development Company**](https://www.leewayhertz.com/hospitality-ai-consulting-and-development-company/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26070354/Group-55.svg)](https://www.leewayhertz.com/ai-use-cases-in-hospitality/)
          * [**AI for Hospitality**](https://www.leewayhertz.com/ai-use-cases-in-hospitality/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26070611/Group-63.svg)](https://www.leewayhertz.com/travel-app-development/)
          * [**Travel App Development**](https://www.leewayhertz.com/travel-app-development/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26070916/Group-64.svg)](https://www.leewayhertz.com/generative-ai-in-travel/)
          * [**GenAI for Travel**](https://www.leewayhertz.com/generative-ai-in-travel/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26071115/Group-60.svg)](https://www.leewayhertz.com/ai-powered-personalized-travel-planning/)
          * [**AI for Personalized Travel**](https://www.leewayhertz.com/ai-powered-personalized-travel-planning/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29030227/Group-61.svg)](https://www.leewayhertz.com/ai-in-customer-support/)
          * [**AI for Customer Support**](https://www.leewayhertz.com/ai-in-customer-support/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29031717/Group-62-2.svg)](https://www.leewayhertz.com/ai-for-sentiment-analysis/)
          * [**AI for Sentiment Analysis**](https://www.leewayhertz.com/ai-for-sentiment-analysis/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29031837/Group-59-1.svg)](https://www.leewayhertz.com/ai-in-demand-forecasting/)
          * [**AI for Demand Forecasting**](https://www.leewayhertz.com/ai-in-demand-forecasting/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29031946/Group-58.svg)](https://www.leewayhertz.com/ai-in-market-research/)
          * [**AI for Market Research**](https://www.leewayhertz.com/ai-in-market-research/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29032305/Group-56-1.svg)](https://www.leewayhertz.com/ai-in-sales/)
          * [**AI for Sales**](https://www.leewayhertz.com/ai-in-sales/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29032533/Group-57-1.svg)](https://www.leewayhertz.com/ai-powered-price-and-promo-optimization-strategies/)
          * [**AI for Price and Promo Optimization**](https://www.leewayhertz.com/ai-powered-price-and-promo-optimization-strategies/)

        *     * [Retail](https://www.leewayhertz.com/retail-software-development/)
      *         *           * [![Retail AI Consulting and Development Company](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Retail AI Consulting and Development Company](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/07/26080201/Retail-AI-Consulting-and-Development-Company.svg)](https://www.leewayhertz.com/retail-ai-consulting-and-development-company/)
          * [**Retail AI Consulting and Development Company**](https://www.leewayhertz.com/retail-ai-consulting-and-development-company/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26064504/Group-54.svg)](https://www.leewayhertz.com/retail-software-development/)
          * [**Retail Software Development**](https://www.leewayhertz.com/retail-software-development/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26064717/Group-49.svg)](https://www.leewayhertz.com/ai-in-retail/)
          * [**AI for Retail**](https://www.leewayhertz.com/ai-in-retail/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26064859/Group-48.svg)](https://www.leewayhertz.com/ai-for-e-commerce/)
          * [**AI for E-commerce**](https://www.leewayhertz.com/ai-for-e-commerce/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26065010/Group-50.svg)](https://www.leewayhertz.com/ai-use-cases-in-fashion/)
          * [**AI for Fashion**](https://www.leewayhertz.com/ai-use-cases-in-fashion/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29033505/Group-51.svg)](https://www.leewayhertz.com/recommender-systems-for-retail-industry-using-ai/)
          * [**Recommender System for Retail**](https://www.leewayhertz.com/recommender-systems-for-retail-industry-using-ai/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29033954/Group-47-1.svg)](https://www.leewayhertz.com/ai-in-store-replenishment/)
          * [**AI for Store Replenishment**](https://www.leewayhertz.com/ai-in-store-replenishment/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29034100/Group-53.svg)](https://www.leewayhertz.com/ai-for-inventory-management-and-demand-forecasting/)
          * [**AI for Inventory Management**](https://www.leewayhertz.com/ai-for-inventory-management-and-demand-forecasting/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29034200/Group-52.svg)](https://www.leewayhertz.com/ai-for-predictive-analytics/)
          * [**AI for Predictive Analytics**](https://www.leewayhertz.com/ai-for-predictive-analytics/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29034314/Group-46-1.svg)](https://www.leewayhertz.com/ai-powered-dynamic-pricing-solution/)
          * [**AI for Dynamic Pricing Solution**](https://www.leewayhertz.com/ai-powered-dynamic-pricing-solution/)

        *         *     * [Healthcare](https://www.leewayhertz.com/healthcare-software-development/)
      *         *           * [![Healthcare AI Consulting and Development Company](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![Healthcare AI Consulting and Development Company](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/08/05042156/healthcare-01.svg)](https://www.leewayhertz.com/healthcare-ai-consulting-and-development-company/)
          * [**Healthcare AI Consulting and Development Company**](https://www.leewayhertz.com/healthcare-ai-consulting-and-development-company/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26041726/Group-30.svg)](https://www.leewayhertz.com/healthcare-software-development/)
          * [**Healthcare Software Development**](https://www.leewayhertz.com/healthcare-software-development/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26042054/Group-31.svg)](https://www.leewayhertz.com/ai-use-cases-in-healthcare/)
          * [**AI for Healthcare**](https://www.leewayhertz.com/ai-use-cases-in-healthcare/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26042303/Group-32.svg)](https://www.leewayhertz.com/ai-in-medicine/)
          * [**AI for Medicine**](https://www.leewayhertz.com/ai-in-medicine/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26043114/Group-75.svg)](https://www.leewayhertz.com/healthcare-fraud-detection-using-ai/)
          * [**AI for Fraud Detection**](https://www.leewayhertz.com/healthcare-fraud-detection-using-ai/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26043340/Group-70.svg)](https://www.leewayhertz.com/generative-ai-in-healthcare/)
          * [**GenAI for Healthcare**](https://www.leewayhertz.com/generative-ai-in-healthcare/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26043728/Group-35.svg)](https://www.leewayhertz.com/generative-ai-in-drug-discovery/)
          * [**AI for Drug Discovery**](https://www.leewayhertz.com/generative-ai-in-drug-discovery/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26043856/Group-38.svg)](https://www.leewayhertz.com/ai-in-telemedicine/)
          * [**AI for Telemedicine**](https://www.leewayhertz.com/ai-in-telemedicine/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26044017/Group-39.svg)](https://www.leewayhertz.com/enterprise-ai-solutions-for-healthcare/)
          * [**Healthcare Enterprise AI Solutions**](https://www.leewayhertz.com/enterprise-ai-solutions-for-healthcare/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26044500/Group-41.svg)](https://www.leewayhertz.com/ai-in-customer-support/)
          * [**AI for Customer Support**](https://www.leewayhertz.com/ai-in-customer-support/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26045726/Group-42.svg)](https://www.leewayhertz.com/ai-for-regulatory-compliance/)
          * [**AI for Regulatory Compliance**](https://www.leewayhertz.com/ai-for-regulatory-compliance/)

        *     * [Consumer Electronics](https://www.leewayhertz.com/consumer-electronics/)
      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26071548/Group-66.svg)](https://www.leewayhertz.com/consumer-electronics/)
          * [**Consumer Electronics Software Development**](https://www.leewayhertz.com/consumer-electronics/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26071714/Group-72.svg)](https://www.leewayhertz.com/ai-for-product-design/)
          * [**AI for Product Design**](https://www.leewayhertz.com/ai-for-product-design/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26071844/Group-71.svg)](https://www.leewayhertz.com/ai-in-product-management/)
          * [**AI for Product Management**](https://www.leewayhertz.com/ai-in-product-management/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26072027/Group-65-1.svg)](https://www.leewayhertz.com/ai-in-production-planning/)
          * [**AI for Production Planning**](https://www.leewayhertz.com/ai-in-production-planning/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29034811/Group-70-1.svg)](https://www.leewayhertz.com/ai-in-product-lifecycle-management/)
          * [**AI for Product Lifecycle Management**](https://www.leewayhertz.com/ai-in-product-lifecycle-management/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29034944/Group-69-1.svg)](https://www.leewayhertz.com/ai-in-market-research/)
          * [**AI for Market Research**](https://www.leewayhertz.com/ai-in-market-research/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29035107/Group-68.svg)](https://www.leewayhertz.com/ai-for-inventory-management-and-demand-forecasting/)
          * [**AI for Inventory Management**](https://www.leewayhertz.com/ai-for-inventory-management-and-demand-forecasting/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29035225/Group-67-2.svg)](https://www.leewayhertz.com/ai-powered-dynamic-pricing-solution/)
          * [**AI Powered Dynamic Pricing**](https://www.leewayhertz.com/ai-powered-dynamic-pricing-solution/)

    * [Startup](https://www.leewayhertz.com/startup-product-development/)
      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26072307/Group-75-1.svg)](https://www.leewayhertz.com/startup-product-development/)
          * [**Startup Product Development**](https://www.leewayhertz.com/startup-product-development/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26072427/Group-76.svg)](https://www.leewayhertz.com/ai-for-startup/)
          * [**AI for Startup**](https://www.leewayhertz.com/ai-for-startup/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26072602/Group-74.svg)](https://www.leewayhertz.com/ai-in-product-development/)
          * [**AI for Product Development**](https://www.leewayhertz.com/ai-in-product-development/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/26072722/Group-73.svg)](https://www.leewayhertz.com/ai-in-product-management/)
          * [**AI for Product Management**](https://www.leewayhertz.com/ai-in-product-management/)

      *         *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29035559/Group-77-1.svg)](https://www.leewayhertz.com/ai-in-product-lifecycle-management/)
          * [**AI for Product Lifecycle Management**](https://www.leewayhertz.com/ai-in-product-lifecycle-management/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29035755/Group-78.svg)](https://www.leewayhertz.com/ai-for-product-design/)
          * [**AI for Product Design**](https://www.leewayhertz.com/ai-for-product-design/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29035909/Group-79-1.svg)](https://www.leewayhertz.com/generative-ai-in-business/)
          * [**GenAI for Business**](https://www.leewayhertz.com/generative-ai-in-business/)

        *           * [![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20186'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/29040030/Group-80.svg)](https://www.leewayhertz.com/ai-driven-development/)
          * [**AI Driven Development**](https://www.leewayhertz.com/ai-driven-development/)

  * [INSIGHTS](https://www.leewayhertz.com/insights/)
  * [WORK](https://www.leewayhertz.com/app-development-work/)
  * [ABOUT](https://www.leewayhertz.com/about-us/)
  * [CONTACT US](https://www.leewayhertz.com/contact-us/)



Select Page

# Prompt engineering: The process, uses, techniques, applications and best practices

[Consult our prompt expert](https://www.leewayhertz.com/contact-us/)

  * [ ![Twitter](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Twitter](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/09204114/xshareicon.svg) Twitter ](http://twitter.com/share?text=Prompt engineering: The process, uses, techniques, applications and best practices&url=https://www.leewayhertz.com/prompt-engineering/ via @leewayhertz)
  * [ ![Facebook](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Facebook](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/09203824/fbshareicon.svg) Facebook ](https://www.facebook.com/sharer/sharer.php?u=https://www.leewayhertz.com/prompt-engineering/)
  * [ ![Linkedin](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Linkedin](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/09203949/linkdinshareicon.svg) Linkedin ](https://www.linkedin.com/shareArticle?mini=true&url=https://www.leewayhertz.com/prompt-engineering/)
  * [ ![Telegram](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Telegram](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/09203718/telegramshareicon.svg) ](https://t.me/share/url?url=https://www.leewayhertz.com/prompt-engineering/)



![prompt-engineering](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/05/29032112/Bridging-the-AI-human-communication-gap-A-comprehensive-guide-to-prompt-engineering-Banner.svg)

**Listen to the article**

What is Chainlink VRF

What is Chainlink VRF

What is Chainlink VRF


In the rapidly evolving artificial intelligence landscape, Large Language Models (LLMs), with OpenAI's ChatGPT at the helm, have achieved remarkable prominence.

The technology demonstrates how the innovative use of language, coupled with computational power, can redefine human-machine interactions. The driving force behind this surge is 'prompt engineering,' an intricate process that involves crafting text prompts to effectively guide LLMs towards accurate task completion, eliminating the need for extra model training.

The effectiveness of [Large Language Models (LLMs)](https://www.leewayhertz.com/large-language-model-development-company/) can be greatly enhanced through carefully crafted prompts. These prompts play a crucial role in extracting superior performance and accuracy from language models. With well-designed prompts, LLMs can bring about transformative outcomes in both research and industrial applications. This enhanced proficiency enables LLMs to excel in a wide range of tasks, including complex question answering systems, arithmetic reasoning algorithms, and numerous others.

However, prompt engineering is not solely about crafting clever prompts. It is a multidimensional field that encompasses a wide range of skills and methodologies essential for the development of robust and effective LLMs and interaction with them. Prompt engineering involves incorporating safety measures, integrating domain-specific knowledge, and enhancing the performance of LLMs through the use of customized prompt engineering tools. These various aspects of prompt engineering are crucial for ensuring the reliability and effectiveness of LLMs in real-world applications.

With a growing interest in unlocking the full potential of LLMs, there is a pressing need for a comprehensive, technically nuanced guide to prompt engineering. In the following sections, we will delve into the core principles of prompting and explore advanced techniques for crafting effective prompts.

  * What is prompt engineering and what are its uses?
  * Importance of prompt engineering in Natural Language Processing (NLP) and artificial intelligence
  * Prompt categories
  * Prompt engineering techniques
    * N-shot prompting
    * Chain-of-thought (CoT) prompting
    * Generated knowledge prompting
    * Directional stimulus prompting
    * ReAct prompting
    * Multimodal CoT prompting
    * Graph prompting
    * Few-shot prompting
    * Self-consistency
    * Prompt chaining
    * Tree of Thoughts (ToT)
    * Automatic Reasoning and Tool-Use (ART)
    * Automatic Prompt Engineer (APE)
    * Active-prompt
  * Prompt engineering: The step-by-step process
    * Understanding the problem
    * Crafting the initial prompt
    * Evaluating the model's response
    * Iterating and refining the prompt
    * Testing the prompt on different models
    * Scaling the prompt
  * Key elements of a prompt
  * How to design prompts?
  * A case study on importance of effective prompt
  * Prompt engineering best practices
  * Applications of prompt engineering
    * Program-aided Language Model (PAL)
    * Generating data
    * Generating code
    * Function calling with LLMs
    * Context caching with Gemini 1.5 Flash
    * Generating synthetic datasets for RAG
    * Enhancing synthetic dataset diversity
    * Case study on prompt engineering for job classification
    * Leveraging encapsulated prompts in GPT-based systems
    * Task decomposition
  * Better prompts, Better results: Tips for successful prompt engineering
  * Risks associated with prompting and solutions
    * Adversarial prompting
    * Factuality
    * Biases
  * Key ethical considerations in prompt engineering for AI models
  * The future of prompt engineering



## What is prompt engineering and what are its uses?

Prompt engineering is the practice of designing and refining specific text prompts to guide transformer-based language models, such as Large Language Models (LLMs), in generating desired outputs. It involves crafting clear and specific instructions and allowing the model sufficient time to process information. By carefully engineering prompts, practitioners can harness the capabilities of LLMs to achieve different goals.

The process of prompt engineering entails analyzing data and task requirements, designing and refining prompts, and fine-tuning the language model based on these prompts. Adjustments to prompt parameters, such as length, complexity, format, and structure, are made to optimize model performance for the specific task at hand.

Professionals in the field of artificial intelligence, including researchers, [data scientists](https://www.leewayhertz.com/hire-data-scientists/), machine learning engineers, and [natural language processing](https://www.leewayhertz.com/natural-language-processing/) experts, utilize prompt engineering to improve the performance and capabilities of LLMs and other AI models. It has applications in various domains, such as improving customer experience in e-commerce, enhancing healthcare applications, and building better conversational AI systems.

Successful examples of prompt engineering include [OpenAI's GPT-3 model](https://www.leewayhertz.com/build-a-gpt-model/) for translation and creative writing, Google's Smart Reply feature for automated message responses, and DeepMind's AlphaGo for playing the game of Go at a high level. In each case, carefully crafted prompts were used to train the models and guide their outputs to achieve specific objectives.

Prompt engineering is crucial for controlling and guiding the outputs of LLMs, ensuring coherence, relevance, and accuracy in generated responses. It helps practitioners understand the limitations of the models and refine them accordingly, maximizing their potential while mitigating unwanted creative deviations or biases.

Isa Fulford and Andrew Ng outlined two principal aspects of prompt engineering in their ChatGPT Prompt Engineering for Developers course:

  1. Formulation of clear and specific instructions: This principle stresses the importance of conciseness and specificity in the prompt construction. Clear prompts assist the model in precisely understanding the required task, thus leading to a more accurate and relevant output.
  2. Allowing the model time to "Think": This principle underscores the significance of giving the model enough time to process the given information. Incorporating pauses or "thinking time" in the prompts can help the model better process and interpret the input, leading to an improved output.



Given the complex nature of [LLMs](https://www.leewayhertz.com/llms-in-synthesizing-training-data/) and their inherent tendency to 'hallucinate,' carefully designed and controlled prompts can help manage these occurrences. Prompt engineering, therefore, plays a crucial role in maximizing the potential of LLMs and mitigating any unwanted creative deviations.

![What is prompt engineering and what are its uses](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201000%20344'%3E%3C/svg%3E)![What is prompt engineering and what are its uses](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/05/28235638/What-is-prompt-engineering-and-what-are-its-uses.svg)

### **Prompts used for various AI tasks**

This section provides examples of how prompts are used for different tasks and introduces key concepts relevant to advanced sections.

Task | Example Prompt | Possible Output  
---|---|---  
Text Summarization | Explain antibiotics. | Antibiotics are medications used to treat bacterial infections…  
Information Extraction | Mention the large language model based product mentioned in the paragraph. | The large language model based product mentioned in the paragraph is ChatGPT.  
Question Answering | Answer the question based on the context below. | It was approved to help prevent organ rejection after kidney transplants.  
Text Classification | Classify the text into neutral, negative, or positive. | Neutral  
Conversation | The following is a conversation with an AI research assistant. | Black holes are regions of spacetime where gravity is extremely strong…  
Code Generation | Ask the user for their name and say "Hello." | let name = prompt("What is your name?"); console.log(`Hello, ${name}!`);  
Reasoning | The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 7, 1. | No, the odd numbers in this group add up to an odd number: 119.  
  
## Contact LeewayHertz’s prompt engineers today!

Enhance the performance of your AI models with our prompt engineering services

[Learn More](https://www.leewayhertz.com/hire-prompt-engineers/?utm_source=prompt-engineering-Insight&utm_medium=CTA)

These examples demonstrate how well-crafted prompts can be utilized for various AI tasks, ranging from text summarization and information extraction to question answering, text classification, conversation, code generation, and reasoning. By providing clear instructions and relevant context in the prompts, we can guide the language model to generate desired outputs.

## Importance of prompt engineering in Natural Language Processing (NLP) and artificial intelligence

In the realm of Natural Language Processing (NLP) and [Artificial Intelligence (AI)](https://www.leewayhertz.com/what-is-artificial-intelligence/), prompt engineering has rapidly emerged as an essential aspect due to its transformative role in optimizing and controlling the performance of language models.

  * **Maximizing model efficiency:** While current transformer-based language models like GPT-3 or Google's PaLM 2 possess a high degree of intelligence, they are not inherently task-specific. As such, they need well-crafted prompts to effectively generate the desired outputs. An intelligently designed prompt ensures that the model's capabilities are utilized optimally, leading to the production of relevant, accurate, and high-quality responses. Thus, prompt engineering allows developers to harness the full potential of these advanced models without the need for extensive re-training or fine-tuning.
  * **Enhancing task-specific performance:** The goal of AI is to enable machines to perform as closely as possible to, or even surpass, human levels. Prompt engineering enables AI models to provide more nuanced, context-aware responses, making them more efficient for specific tasks. Whether it's language translation, sentiment analysis, or text generation, prompt engineering helps to align the model's output with the task's requirements.
  * **Understanding model limitations:** Working with prompts can provide insight into the limitations of a language model. Through iterative refining of prompts and studying the responses of the model, we can understand its strengths and weaknesses. This knowledge can guide future model development, feature enhancement, and even lead to new approaches in NLP.
  * **Increasing model safety:** AI safety is an important concern, especially when using language models for public-facing applications. A poorly designed prompt might lead the model to generate inappropriate or harmful content. Skilled prompt engineering can help prevent such issues, making the AI model safer to use.
  * **Enabling resource efficiency:** Training large language models demands considerable computational resources. However, with effective prompt engineering, developers can significantly improve the performance of a [pre-trained model](https://www.leewayhertz.com/fine-tuning-pre-trained-models/) without additional resource-intensive training. This not only makes the AI development process more resource-efficient but also more accessible to those with limited computational resources.
  * **Facilitating domain-specific knowledge transfer:** Through skilled prompt engineering, developers can imbue language models with domain-specific knowledge, allowing them to perform more effectively in specialized fields such as medical, legal, or technical contexts.



In a nutshell, prompt engineering is crucial for the effective utilization of large language models in NLP-based and other tasks, helping to maximize model performance, ensure safety, conserve resources, and improve domain-specific outputs. As we move forward into an era where AI is increasingly integrated into daily life, the importance of this field will only continue to grow.

## Prompt categories

Prompts play a crucial role in fostering efficient interaction with AI language models. The fundamental aspect of crafting proficient prompts lies in comprehending their diverse types. This comprehension greatly facilitates the process of tailoring prompts to elicit a specific desired response.

There are several primary categories of prompts:

  * **Queries for information -** These prompts are tailored to obtain information. They generally respond to the questions "What" and "How." Instances of such prompts are: "What are the top places to visit in Kenya?", "How can I prepare for a job interview?"
  * **Task-specific prompts -** They are employed to direct the model to accomplish a certain task. These types of prompts are commonly seen in use with Siri, Alexa, or Google Assistant. For instance, a task-specific prompt could be "Dial mom" or "Begin playing the newest episode of my preferred TV series."
  * **Context-supplying prompts -** As the name implies, these prompts offer details to the AI to help it accurately discern the user's requirements. For instance, if you're organizing a party and need some inspiration for decorations and activities, you could structure your prompt like this: "I'm organizing a party for my child, could you suggest some decoration ideas and activities to make it fun and memorable?"
  * **Comparative prompts -** These are utilized to assess or compare different choices given to the model to assist the user in making an appropriate decision. For instance: "How does Option A stack up against Option B in terms of strengths and weaknesses?"
  * **Opinion-eliciting prompts -** These are crafted to solicit the AI's viewpoint on a particular subject. For instance: "What might occur if time travel were possible?"
  * **Reflective prompts -** These prompts are designed to help individuals delve deeper into understanding themselves, their beliefs, and their actions. They are akin to prompts for self-improvement based on a topic or personal experience. A bit of prior information might be required to get the desired response.
  * **Role-specific prompts -** Role prompting is a technique where the model is instructed to take on a specific role before engaging in a particular task. It involves using a prompt to define the AI's role, such as a doctor or lawyer, and then asking questions or providing scenarios related to that role. This is the most frequently employed category of prompts. Assigning a role to the AI produces responses based on that role. A useful strategy for this category is to employ the 5 Ws framework, which includes: 
    * **Who -** Assigns the role you want the model to assume. A role such as a teacher, developer, chef, etc.
    * **What -** Refers to the action you want the model to perform.
    * **When -** Your desired timeframe for accomplishing a particular task.
    * **Where -** Refers to the location or context of a specific prompt.
    * **Why -** Refers to a specific prompt's reasons, motives, or objectives.
  * **Classification prompts -** These prompts are designed to classify input text into categories based on predefined criteria. 
    * **Sentiment classification:** Determines the sentiment of a given text (e.g., positive, negative, neutral). 
      * **Example:** "Classify the sentiment of the following review: 'I loved the new phone! It has great features and a sleek design.'"
    * **Few-shot sentiment classification:** Uses a few examples to help the model understand how to classify sentiment. 
      * **Example:** "Classify the sentiment of the following review based on these examples: Positive: 'The service was excellent.' Negative: 'The food was cold and tasteless.' Neutral: 'The product arrived on time.' Review: 'The staff were friendly but the service was slow.'"
  * **Coding prompts -** Coding prompts guide the model in generating code snippets or queries for specific programming tasks. These prompts help automate and streamline coding tasks by providing instructions for code creation. For example, you might use a prompt like "Generate a Python function that calculates the factorial of a number" to produce a functional piece of code, or "Generate a MySQL query to retrieve all customer names from the 'customers' table where the 'age' is greater than 30" to create a database query. Additionally, prompts such as "Generate a TiKZ diagram of a simple flowchart with three steps: Start, Process, End" can be used to create visual diagrams in LaTeX.
  * **Creativity prompts -** Creativity prompts are designed to inspire imaginative outputs and novel ideas by encouraging the generation of unique and original content. They can cover a wide range of creative tasks, from producing rhyming poetry to inventing new concepts or words. For example, you might ask for a short poem that rhymes about a sunny day, generate an infinite sequence of prime numbers, or invent a new word for a hybrid device combining a phone and a tablet. These prompts are ideal for exploring creative expression and innovative thinking.
  * **Evaluation prompts -** Evaluation prompts are used to assess, analyze, or critique specific content, offering insights or detailed evaluations of various subjects. They are useful for providing in-depth analysis or critique of complex materials. For instance, you might be asked to evaluate the main arguments presented in Plato's "The Republic" regarding justice or assess the effectiveness of a particular philosophical argument. These prompts help in critically examining and understanding the strengths and weaknesses of the content.
  * **Image generation prompts -** Image generation prompts guide the model to create visual content based on descriptive text, transforming written instructions into visual representations. For example, you can use these prompts to generate an ASCII art representation of a person using specific letters and symbols. An example prompt could be, "Draw a simple ASCII art of a person using the letters 'A,' 'B,' and 'C.'" These prompts allow for the creation of visual elements from textual descriptions, making them versatile tools for various creative tasks.
  * **Question answering prompts -** Question answering prompts are designed to provide responses to inquiries across various domains. These prompts can be tailored for specific contexts or broader topics, making them versatile for different needs. For instance, closed domain question answering targets specific areas such as, "What is the capital of France?" focusing on geography. Open domain question answering covers a wide range of subjects, like, "Who was the first person to walk on the moon?" While science question answering delves into scientific concepts, for example, "Explain the theory of relativity." Each type of prompt helps in addressing specific types of queries efficiently.
  * **Reasoning prompts -** Reasoning prompts require the application of logical or physical principles to derive answers or solve problems. They can involve indirect reasoning, where conclusions are drawn from implied or indirect information, such as, "If all humans are mammals and some mammals are not dogs, what can we infer about humans and dogs?" They also include physical reasoning, which applies physical laws or principles, such as, "If you drop a ball from a height, how does gravity affect its fall?" These prompts are essential for evaluating and understanding complex scenarios through reasoning.
  * **Text summarization prompts -** Text summarization prompts are designed to condense or clarify information from a given text, making complex ideas more understandable. They often involve explaining specific concepts, such as, "Explain the concept of quantum entanglement in simple terms." These prompts help distill detailed information into accessible summaries, aiding in comprehension and retention.
  * **Truthfulness prompts -** Truthfulness prompts are designed to evaluate and ensure the accuracy of information. They focus on identifying and correcting false or misleading statements, such as, "Identify and correct any inaccuracies in the following statement: 'The Great Wall of China is visible from the moon.'" These prompts help verify the reliability of content and maintain factual accuracy.
  * **Adversarial prompts -** Adversarial prompting involves testing the model’s resilience against challenging or deceptive inputs. These prompts aim to assess how well the model handles difficult scenarios and potential manipulations. For example, you might ask, "Provide a response to this misleading question: 'If I say I’m lying, am I telling the truth?'" to evaluate the model's ability to navigate complex and ambiguous queries.



## Prompt engineering techniques

Prompt engineering, an emergent area of research that has seen considerable advancements since 2022, employs a number of novel techniques to enhance the performance of language models. Each of these techniques brings a unique approach to instructing large language models, highlighting the versatility and adaptability inherent in the field of prompt engineering. They form the foundation for effectively communicating with these models, shaping their output, and harnessing their capabilities to their fullest potential. Some of the most useful methods widely implemented in this field are:

### N-shot prompting (zero-shot prompting and few-shot prompting)

The term "N-shot prompting" is used to represent a spectrum of approaches where N symbolizes the count of examples or cues given to the language model to assist in generating predictions. This spectrum includes, notably, zero-shot prompting and few-shot prompting.

  * **Zero-shot prompting** refers to a situation where the language model generates predictions without any explicit, additional examples. It's particularly effective for tasks the model has been extensively trained on, including but not limited to, classification tasks like [sentiment analysis](https://www.leewayhertz.com/ai-for-sentiment-analysis/) or spam detection, text transformation tasks like translation or summarization, and simple text generation.



Let's consider the task of sentiment analysis where the model is given a sentence and is expected to classify the sentiment as positive, negative, or neutral. In this case, we could directly ask the model without providing any explicit training examples. This would be an instance of zero-shot prompting. Here is how it might look:

Input prompt to the model:
    
    
    "What is the sentiment of the following sentence: 'I had an amazing day at the park'?"

Given that the language model has been extensively trained on understanding sentiments during its initial training, it can typically respond to such a prompt accurately, despite not receiving specific examples in this context. The expected response would be:

Model's response:
    
    
    "The sentiment of the sentence is positive."

In this scenario, we haven't provided any training examples or clues. We are directly asking the model to perform the task, hence it's a zero-shot prompt.

  * On the other hand, **few-shot prompting** employs a limited set of examples, usually ranging from two to five, to guide the model's output. These instances are designed to direct the model towards improved performance in addressing more context-specific problems. By offering a snapshot of the desired output, few-shot prompts enable the model to tailor its responses more effectively, thus increasing the accuracy of its predictions.



Let's consider the task of generating a rhymed couplet (two-line verse) about a given topic, which is a more context-specific task and might benefit from few-shot prompting. Here is how it might look:

Input prompt to the model:
    
    
    "Write a rhymed couplet about a sunflower:
    Example 1:
    'Sunflower with petals bright,
    Basking gladly in the sunlight.'
    Example 2:
    'Sunflower tall in the summer glow,
    Nodding as the breezes blow.'
     
    Now, write a rhymed couplet about a moonlit night."

Here, we are providing the model with two examples of how to write a rhymed couplet about a specific topic, in this case, a sunflower. These examples serve as context and steer the model towards the desired output. They essentially "tell" the model what we expect from it.

The model, given its extensive training data and the two specific examples (few-shot prompting), is expected to generate a similar rhymed couplet about a moonlit night. The response could be:

Model's response:
    
    
    "Moonlight spreading its silver light,
    Bathing the world in a tranquil night."

In this scenario, we have provided a few examples or clues before asking the model to perform the task, hence it's a few-shot prompt.

### Chain-of-thought (CoT) prompting

Chain-of-thought (CoT) prompting, a technique introduced by Google researchers, operates on the concept of encouraging an AI model to elucidate intermediate reasoning stages before delivering the final answer to a multi-stage issue. The objective is to design the model's reasoning trajectory to resemble the intuitive cognitive process one would employ while tackling a complex problem involving multiple steps. This procedure allows the model to dissect intricate problems into simpler components, thereby enabling it to address challenging reasoning tasks that traditional prompting methods might not handle effectively.

Let's consider a complex problem-solving example in which Chain-of-thought (CoT) prompting can be applied.

Consider a prompt where we want a language model to solve a multi-step math word problem like this:
    
    
    "John has 10 apples. He gives 3 apples to his friend Sam and then buys 6 more apples from the market. How many apples does John have now?"
    

Using Chain-of-thought prompting, we would split the problem into simpler intermediate steps:
    
    
    Initial Prompt: "John has 10 apples." Intermediate Prompt: "How many apples does John have if he gives 3 to Sam?" Intermediate Answer: "John has 7 apples."
     
    Initial Prompt: "John has 7 apples." Intermediate Prompt: "How many apples will John have if he buys 6 more apples from the market?" Intermediate Answer: "John has 13 apples."

Finally, we have the answer to the original complex problem: "John has 13 apples now."

The chain-of-thought prompting method breaks down the problem into manageable pieces, allowing the model to reason through each step and then build up to the final answer. This method helps to increase the model's problem-solving capabilities and overall understanding of complex tasks.

There are several innovative adaptations of chain-of-thought prompting, including:

  * **Self-consistency prompting:** This variation involves creating multiple diverse paths of reasoning and selecting answers that show the highest level of consistency. This method ensures increased precision and dependability in answers by implementing a consensus-based system.



Let's consider an example where we use self-consistency prompting in a scenario involving decision-making based on diverse reasoning paths.

Let's assume we are asking a language model to predict the best holiday destination considering the following variables: budget, proximity, weather, and activities. The resulting question could be:
    
    
    "What would be the best holiday destination considering a budget of $3000, a preference for warm weather, and a love for beach activities?"

We then sample diverse reasoning paths:
    
    
    Path 1: Initial prompt: "Considering a budget of $3000 and a preference for warm weather." Intermediate answer: "A tropical destination would fit these criteria. Let's consider Bali."
     
    Following prompt: "What about considering beach activities?" Final answer: "Bali is renowned for its beach activities. So, it could be the best choice."
     
    Path 2: Initial prompt: "Considering a preference for warm weather and a love for beach activities." Intermediate answer: "Hawaii is well-known for its excellent weather and beach activities."
     
    Following prompt: "Considering a budget of $3000, is Hawaii a good choice?" Final answer: "Yes, Hawaii could be a great choice considering all three factors."
     
    Path 3: Initial prompt: "Considering a budget of $3000 and a love for beach activities." Intermediate answer: "Considering these factors, the Bahamas could be a great destination."

Following prompt:
    
    
    "Considering a preference for warm weather, is the Bahamas a good choice?"

Final answer:
    
    
    "Indeed, the Bahamas is known for its warm weather. So, it could be the best choice."

In this case, all paths led to valid holiday destinations. To make a decision, we then apply a majority voting system, wherein the most consistent answer would be chosen as the final output of the self-consistency prompting process. Given the diversity of the prompts, the most consistent destination can be considered the most suitable for the given conditions.

  * **Least-to-most prompting (LtM):** Here, the chain-of-thought technique begins by fragmenting a problem into a series of less complex sub-problems. The model then solves them in an ordered sequence. Each subsequent sub-problem is solved using the solutions to previously addressed sub-problems. This methodology is motivated by real-world teaching strategies used in educating children.



Let's consider an example where we use least-to-most prompting in the context of solving a mathematical word problem. The problem is: "John has twice as many apples as Jane. Jane has 5 apples. How many apples does John have?"

In the least-to-most prompting approach, we would break down this problem into simpler subproblems and solve them sequentially.

First subproblem: Initial prompt:
    
    
    "Jane has 5 apples." Intermediate answer: "So, the number of apples Jane has is 5."

Second Subproblem: Initial prompt:
    
    
    "John has twice as many apples as Jane."

Intermediate answer:
    
    
    "So, John has 2 times the number of apples that Jane has."
    

Third Subproblem: Initial prompt:
    
    
    "Given that Jane has 5 apples and John has twice as many apples as Jane, how many apples does John have?"

Final answer:
    
    
    "John has 2 * 5 = 10 apples."

In this way, the least-to-most prompting technique decomposes a complex problem into simpler subproblems and builds upon the answers to previously solved subproblems to arrive at the final answer.

  * **Active prompting:** This technique scales the CoT approach by identifying the most crucial and beneficial questions for human annotation. Initially, the model computes the uncertainty present in the LLM’s predictions, then it selects the questions that contain the highest uncertainty. These questions are sent for human annotation, after which they are integrated into a CoT prompt.



Active prompting involves identifying and selecting uncertain questions for human annotation. Let's consider an example from the perspective of a language model engaged in a conversation about climate change.

Let's assume our model has identified three potential questions that could be generated from its current conversation, with varying levels of uncertainty:

  1. What is the average global temperature?
  2. What are the primary causes of global warming?
  3. How does carbon dioxide contribute to the greenhouse effect?



In this scenario, the model might be relatively confident about the answers to the first two questions, since these are common questions about the topic. However, it might be less certain about the specifics of how carbon dioxide contributes to the greenhouse effect.

Active prompting would identify the third question as the most uncertain, and thus most valuable for human annotation. After this question is selected, a human would provide the model with the information required to correctly answer the question. The annotated question and answer would then be added to the model's prompt, enabling it to better handle similar questions in the future.

## Contact LeewayHertz’s prompt engineers today!

Enhance the performance of your AI models with our prompt engineering services

[Learn More](https://www.leewayhertz.com/hire-prompt-engineers/?utm_source=prompt-engineering-Insight&utm_medium=CTA)

### Generated knowledge prompting

Generated knowledge prompting operates on the principle of leveraging a large language model's ability to produce potentially beneficial information related to a given prompt. The concept is to let the language model offer additional knowledge which can then be used to shape a more informed, contextual, and precise final response.

For instance, if we are using a language model to provide answers to complex technical questions, we might first use a prompt that asks the model to generate an overview or explanation of the topic related to the question.

Suppose the question is: "Can you explain how quantum entanglement works in quantum computing?"

We might first prompt the model with a question like, "Provide an overview of quantum entanglement." The model might generate a response detailing the basics of quantum entanglement.

We would then use this generated knowledge as part of our next prompt. We might ask: "Given that quantum entanglement involves the instantaneous connection between two particles regardless of distance, how does this concept apply in quantum computing?"

By using generated knowledge prompting in this way, we are able to facilitate more informed, accurate, and contextually aware responses from the language model.

### Directional stimulus prompting

Directional stimulus prompting is another advanced technique in the field of prompt engineering where the aim is to direct the language model's response in a specific manner. This technique can be particularly useful when you are seeking an output that has a certain format, structure, or tone.

For instance, suppose you want the model to generate a concise summary of a given text. Using a directional stimulus prompt, you might specify not only the task ("summarize this text") but also the desired outcome, by adding additional instructions such as "in one sentence" or "in less than 50 words". This helps to direct the model towards generating a summary that aligns with your requirements.

Here is an example: Given a news article about a new product launch, instead of asking the model "Summarize this article," you might use a directional stimulus prompt such as "Summarize this article in a single sentence that could be used as a headline."

Another example could be in generating rhymes. Instead of asking, "Generate a rhyme," a directional stimulus prompt might be, "Generate a rhyme in the style of Dr. Seuss about friendship."

By providing clear, specific instructions within the prompt, directional stimulus prompting helps guide the language model to generate output that aligns closely with your specific needs and preferences.

### ReAct prompting

ReAct prompting is a technique inspired by the way humans learn new tasks and make decisions through a combination of "reasoning" and "acting". This innovative methodology seeks to address the limitations of previous methods like Chain-of-thought (CoT) prompting, which, despite its ability to generate reasonable answers for various tasks, has issues related to fact hallucination and error propagation due to its lack of interaction with external environments and inability to update its knowledge.

ReAct prompting pushes the boundaries of large language models by prompting them to not only generate verbal reasoning traces but also actions related to the task at hand. This hybrid approach enables the model to dynamically reason and adapt its plans while interacting with external environments, such as databases, APIs, or in simpler cases, information-rich sites like Wikipedia.

For example, if we task an LLM with the goal of creating a detailed report on the [current state of artificial intelligence](https://www.leewayhertz.com/current-state-of-generative-ai/), using ReAct prompting, the model would not just generate responses based on its pre-existing knowledge. Instead, it would plan a sequence of actions, such as fetching the latest AI research papers from a database or querying for recent news on AI from reputable sources. It would then integrate this up-to-date information into its reasoning process, resulting in a more accurate and comprehensive report. This two-pronged approach of acting and reasoning can mitigate the limitations observed in prior prompting methods and empower LLMs with enhanced accuracy and depth.

Consider a scenario where a user wants to know the current state of a particular stock. Using the ReAct prompting technique, the task might unfold in the following steps:

  1. **Step 1 (Reasoning)** : The LLM determines that to fulfill this request, it needs to fetch the most recent stock information. The model identifies the required action, i.e., accessing the latest stock data from a reliable financial database or API.
  2. **Step 2 (Acting)** : The model generates a command to retrieve the data: "Fetch latest stock data for 'Company X' from the Financial Database API".
  3. **Step 3 (Interaction)** : The command is executed, and the model receives the up-to-date stock information.
  4. **Step 4 (Reasoning and Acting)** : With the latest stock data now available, the model processes this information and generates a detailed response: "As of today, the stock price of 'Company X' is at $Y, which represents a Z% increase from last week."



In this example, the LLM demonstrates its ability to reason and generate actions (fetching the data), interact with an external environment (the financial database API), and ultimately provide a precise and informed response based on the most recent data available.

### Multimodal CoT prompting

Multimodal CoT prompting is an extension of the original CoT prompting, involving multiple modes of data, usually both text and images. By using this technique, a large language model can leverage visual information in addition to text to generate more accurate and contextually relevant responses. This allows the system to carry out more complex reasoning that involves both visual and textual data.

For instance, consider a scenario where a user wants to know the type of bird shown in a particular image. Using the [multimodal](https://www.leewayhertz.com/multimodal-model/) CoT prompting technique, the task might unfold as follows:

  1. **Step 1 (Reasoning)** : The LLM recognizes that it needs to identify the bird in the image. However, instead of making a direct guess, it decides to carry out a sequence of reasoning steps, first trying to identify the distinguishing features of the bird.
  2. **Step 2 (Acting)** : The model generates a command to analyze the image: "Analyze the bird's features in the image, such as color, size, and beak shape."
  3. **Step 3 (Interaction)** : The command is executed, and the model receives the visual analysis of the bird: "The bird has blue feathers, a small body, and a pointed beak."
  4. **Step 4 (Reasoning and Acting)** : With these distinguishing features now available, the model cross-references this information with its textual knowledge about bird species. It concludes that the bird is likely to be a "Blue Tit."
  5. **Step 5 (Final Response)** : The model provides its final answer: "Based on the blue feathers, small body, and pointed beak, the bird in the image appears to be a Blue Tit."



In this example, multimodal CoT prompting allows the LLM to generate a chain of reasoning that involves both image analysis and textual cross-referencing, leading to a more informed and accurate answer.

### Graph prompting

Graph prompting is a method for leveraging the structure and content of a graph for prompting a large language model. In graph prompting, you use a graph as the primary source of information and then translate that information into a format that can be understood and processed by the LLM. The graph could represent many types of relationships, including social networks, biological pathways, and organizational hierarchies, among others.

For example, let us consider a graph that represents relationships between individuals in a social network. The nodes of the graph represent people, and the edges represent relationships between them. Let us say you want to find out who in the network has the most connections.

You would start by translating the graph into a textual description that an LLM can process. This could be a list of relationships like "Alice is friends with Bob," "Bob is friends with Charlie," "Alice is friends with Charlie," and so on.

Next, you would craft a prompt that asks the LLM to analyze these relationships and identify the person with the most connections. The prompt might look like this: "Given the following list of friendships, who has the most friends: Alice is friends with Bob, Bob is friends with Charlie, Alice is friends with Charlie."

The LLM would then process this prompt and provide an answer based on its analysis of the information. For instance, in this case, the answer might be "Alice", given that she has the most connections according to the provided list of relationships.

Through graph prompting, you are essentially converting structured graph data into a text-based format that LLMs can understand and reason about, opening up new possibilities for question answering and problem solving.

### Few-shot prompting

Few-shot prompting is a technique used to enhance the performance of large language models (LLMs) on more complex tasks by providing them with specific examples within the prompt. Unlike zero-shot prompting, which relies solely on the model’s general training, few-shot prompting incorporates demonstrations to guide the model towards generating better responses. These examples serve as contextual cues, helping the model understand how to handle similar tasks effectively.

The concept of few-shot prompting gained prominence with the scaling of models, as noted by [Touvron et al. (2023)](https://arxiv.org/pdf/2302.13971) and [Kaplan et al. (2020)](https://arxiv.org/abs/2001.08361). The technique leverages in-context learning, where the model learns from provided examples to perform a specific task. This can be particularly useful when dealing with tasks that are too complex for the model to handle effectively in a zero-shot setting.

**Example of Few-shot prompting:**

**Prompt:**  
A "whatpu" is a small, furry animal native to Tanzania. An example of a sentence that uses the word whatpu is:  
We were traveling in Africa and we saw these very cute whatpus.

To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is:

**Output:**  
When we won the game, we all started to farduddle in celebration.

In this example, the model was given one demonstration (1-shot) of how to use a new word in a sentence and successfully applied the concept to a new term. For more complex tasks, increasing the number of examples (e.g., 3-shot, 5-shot) can further improve performance.

According to research by [Min et al. (2022)](https://arxiv.org/abs/2202.12837), several factors influence the effectiveness of few-shot prompting:

  * Label Space and Input Distribution: The range of labels and the distribution of input text provided in examples are crucial. Even if labels are not perfectly aligned, a well-structured format helps.
  * Format Consistency: Consistent formatting, even with randomized labels, improves performance compared to having no labels at all. Selecting labels from a true distribution rather than a uniform one also enhances results.



**Example with random labels:**

**Prompt:**  
This is awesome! // Negative  
This is bad! // Positive  
Wow that movie was rad! // Positive  
What a horrible show! //

**Output:**  
Negative

The model correctly identifies the sentiment despite the random labeling of examples. Even when the format varies, as shown in the following example, recent models display robustness to different formats:

**Example with Varying Format:**

**Prompt:**  
Positive This is awesome!  
This is bad! Negative  
Wow that movie was rad!  
Positive  
What a horrible show! --

**Output:**  
Negative

Although the format is inconsistent, the model still predicts the correct label. This demonstrates the model's growing ability to handle diverse formats and random labels, although further analysis is needed to confirm effectiveness across various tasks and prompt variations.

### Self-consistency

Self-consistency is an advanced prompt engineering technique designed to enhance the reliability of model outputs, particularly in tasks involving complex reasoning such as arithmetic and commonsense challenges. Introduced by Wang et al. (2022), this method builds upon chain-of-thought (CoT) prompting by addressing its limitations with a more robust approach to generating consistent results.

The core idea behind self-consistency is to improve upon the basic greedy decoding method used in CoT prompting. Instead of relying on a single reasoning path, self-consistency involves sampling multiple diverse reasoning paths for a given task. By generating several possible solutions or responses, the technique then evaluates which of these is the most consistent across different paths.

This approach is particularly effective in tasks that require detailed reasoning and multi-step processes. For example, when dealing with arithmetic problems or tasks requiring commonsense reasoning, self-consistency helps by aggregating multiple responses and selecting the one that consistently aligns with the most frequent or coherent solution.

**Example of self-consistency in arithmetic reasoning:**

Let's use self-consistency to solve a problem involving basic arithmetic operations:

**Initial prompt:**  
A company had 120 employees at the start of the year. During the year, they hired 45 new employees and 30 employees left the company. How many employees are currently working at the company?

**Initial output:**  
135

This output is incorrect. To improve accuracy using self-consistency, we generate several different reasoning paths:

**Prompts with few-shot exemplars:**

Q: A library had 200 books. They received 50 new books and gave away 30 books. How many books are now in the library?  
A: The library started with 200 books. After receiving 50 new books and giving away 30, the total number of books is 200 + 50 - 30 = 220. The answer is 220.

Q: There are 10 students in a class. If 5 more students join the class and 2 students leave, how many students are there now?  
A: The class started with 10 students. After adding 5 students and removing 2, the total number of students is 10 + 5 - 2 = 13. The answer is 13.

**Applying self-consistency to the original task:**

Using different reasoning paths, we generate multiple responses:  
Output 1: The company had 120 employees initially. They hired 45 more employees, making it 120 + 45 = 165 employees. With 30 employees leaving, the total number is 165 - 30 = 135 employees. The answer is 135.

Output 2: Starting with 120 employees, the company hired 45 new employees, bringing the total to 120 + 45 = 165. After 30 employees left, the final count is 165 - 30 = 135. The answer is 135.

Output 3: Initially, there were 120 employees. Adding 45 new employees results in 120 + 45 = 165. Subtracting the 30 employees who left, the total is 165 - 30 = 135 employees. The answer is 135.

In this case, all generated outputs are consistent and agree on the final number of 135 employees. Although all the answers match, the self-consistency approach ensures reliability by checking the agreement among multiple reasoning paths.

Self-consistency helps to solidify the accuracy of responses by considering various paths and ensuring that the final answer is robust across different reasoning approaches.

### Prompt chaining

Prompt chaining is a sophisticated technique used to enhance the reliability and performance of large language models (LLMs). It involves breaking a complex task into smaller, manageable subtasks and sequentially processing them through a series of interconnected prompts. Each prompt in the chain builds upon the output of the previous one, allowing the model to handle intricate queries more effectively than when approached with a single, detailed prompt.

**How prompt chaining works:**

In prompt chaining, a complex task is decomposed into several subtasks, each addressed by a different prompt. The output of one prompt serves as the input for the next, creating a sequence or "chain" of prompt operations. This method allows for more granular control over the model's responses and can improve accuracy, transparency, and debugging capabilities.

**Benefits of prompt chaining:**

  * **Improved performance:** By breaking down tasks, prompt chaining helps LLMs manage complex queries more effectively.
  * **Enhanced transparency:** It allows for easier analysis and debugging by examining the performance at each stage of the chain.
  * **Greater control:** Adjustments can be made to individual prompts to refine the overall performance of the system.
  * **Application in conversational assistants:** It aids in building more effective and personalized conversational agents.



**Example of prompt chaining for document question answering:**

Prompt chaining is particularly useful for tasks that involve multiple steps, such as answering questions based on a large document. Here’s an example of how prompt chaining can be applied in such a scenario:

#### Step 1: Extract relevant quotes

The first prompt is designed to extract relevant quotes from a document based on a specific question.

**Prompt 1:**  
Given the document below and the question, extract relevant quotes that help answer the question.

Document:  
{{document}}

Question:  
What are the key techniques for prompting in large language models?

**Output of prompt 1:**  
<quotes>  
- Chain-of-thought (CoT) prompting  
- Self-consistency decoding  
- Generated knowledge prompting  
- Textual inversion and embeddings  
</quotes>

#### Step 2: Compose the final answer

The second prompt takes the extracted quotes and the original document to formulate a final response.

**Prompt 2:**  
Given the following quotes (delimited by <quotes></quotes>) extracted from a document and the original document (delimited by ####), please compose an accurate and helpful answer to the question.

####  
{{document}}  
####

<quotes>  
- Chain-of-thought (CoT) prompting  
- Self-consistency decoding  
- Generated knowledge prompting  
- Textual inversion and embeddings  
</quotes>

**Output of prompt 2:**  
The document discusses several key techniques for prompting in large language models, including:

1\. Chain-of-thought (CoT) prompting  
2\. Self-consistency decoding  
3\. Generated knowledge prompting  
4\. Textual inversion and embeddings

These techniques each use unique strategies to enhance the interaction with language models and improve the performance of the system.

By using prompt chaining, the model is guided through a structured process that breaks down the task into smaller steps, leading to more accurate and coherent results. This technique is valuable for tasks requiring complex reasoning or multiple operations, and it can be adapted to various applications, including conversational assistants and document-based question-answering.

### Tree of Thoughts (ToT)

When addressing complex tasks that involve exploration and strategic decision-making, traditional prompting techniques may not suffice. The Tree of Thoughts (ToT) framework, introduced by [Yao et al. (2023)](https://arxiv.org/abs/2305.10601) and [Long (2023)](https://arxiv.org/abs/2305.08291), offers a more sophisticated approach by building upon and extending chain-of-thought prompting. This framework facilitates a structured exploration of intermediate steps, enabling more effective problem-solving with language models.

**Overview of Tree of Thoughts (ToT):**

ToT operates by maintaining a "tree" of thoughts, where each thought represents a coherent sequence of language that contributes to solving a problem. This tree structure allows the language model (LM) to generate and evaluate a series of intermediate thoughts systematically, employing search algorithms to guide the exploration process.

**Key components of ToT:**

  1. **Tree structure:** Each node in the tree represents an intermediate thought or step in the problem-solving process. This structure supports systematic exploration and evaluation of potential solutions.
  2. **Search algorithms:** ToT integrates search techniques such as breadth-first search (BFS) and depth-first search (DFS) to explore the tree. These algorithms help in evaluating different paths and making decisions based on lookahead and backtracking.
  3. **Evaluation mechanism:** As thoughts are generated, they are evaluated with labels such as "sure," "maybe," or "impossible." This evaluation guides the search process by promoting feasible partial solutions and discarding those that are less likely to contribute to the final solution.
  4. **Adaptive search strategies:** [Yao et al. (2023)](https://arxiv.org/abs/2305.10601) and [Long (2023)](https://arxiv.org/abs/2305.08291) both utilize tree search strategies, but with variations. Yao et al. leverage traditional search methods like DFS/BFS/beam search, while Long (2023) introduces a "ToT Controller" trained through reinforcement learning (RL) to adapt and optimize the search strategy over time.



**Example use case: Mathematical reasoning**

In the Game of 24, a mathematical reasoning task, ToT can decompose the problem into multiple steps. For instance, each step involves evaluating intermediate equations to reach the target number. The LM generates and evaluates several candidate solutions at each step, retaining the best options based on the evaluation criteria.

**Prompting with Tree of Thoughts:**

A simplified version of ToT, known as Tree-of-Thought Prompting, has been proposed by Hulbert (2023). This technique involves simulating a panel of experts who collaboratively solve a problem step-by-step:

**Prompt example:**  
Imagine three different experts are answering this question.  
All experts will write down 1 step of their thinking,  
then share it with the group.  
Then all experts will go on to the next step, etc.  
If any expert realizes they’re wrong at any point, they leave.  
The question is…

This approach allows the LM to simulate a structured and iterative problem-solving process, enhancing the reliability and depth of the responses.

**Recent developments:**

[Sun (2023)](https://github.com/holarissun/PanelGPT) further refined the Tree-of-Thought technique by introducing PanelGPT, a method that uses panel discussions among multiple LLMs to tackle complex problems. This approach leverages collaborative reasoning among models to enhance problem-solving capabilities.

Tree of Thoughts (ToT) provides a robust framework for handling complex tasks by leveraging a structured approach to intermediate reasoning steps. By integrating search algorithms and adaptive strategies, ToT enhances the problem-solving capabilities of language models, making it a valuable technique for tasks requiring strategic exploration and evaluation.

### Automatic Reasoning and Tool-Use (ART)

In the quest to enhance the capabilities of large language models (LLMs), integrating reasoning with tool usage has emerged as a promising approach. Traditionally, this has involved manually crafting task-specific demonstrations and scripting intricate interactions between model generations and external tools. However, a new framework introduced by [Paranjape et al. (2023)](https://arxiv.org/abs/2303.09014), known as Automatic Reasoning and Tool-Use (ART), offers a more automated and flexible solution.

**Overview of Automatic Reasoning and Tool-Use (ART):**

ART is designed to combine chain-of-thought (CoT) prompting with the utilization of external tools in a seamless, automated manner. This approach aims to address the complexities of multi-step reasoning tasks by leveraging both pre-existing task libraries and dynamic tool integration.

**How ART works?**

  1. **Task selection and demonstrations:** When a new task is presented, ART begins by selecting relevant demonstrations of multi-step reasoning and tool usage from a pre-defined task library. These demonstrations serve as models for how to approach similar tasks.
  2. **Interleaved reasoning and tool use:** During the task execution phase, ART automates the process of generating intermediate reasoning steps. Whenever an external tool is needed, ART pauses the model’s generation, integrates the tool’s output, and then resumes the reasoning process. This interleaved approach ensures that prompt engineering tools are used appropriately and effectively throughout the task.
  3. **Zero-shot generalization:** ART’s framework allows models to generalize from the provided demonstrations to decompose new tasks and use tools appropriately without requiring explicit training on each specific task. This zero-shot capability is a significant advancement over traditional methods.
  4. **Extensibility and human interaction:** ART is designed to be flexible and extensible. Users can update the task and tool libraries to correct reasoning errors or introduce new tools. This adaptability helps maintain the relevance and accuracy of the system’s performance.



**Performance and advantages:**

  1. **Improved accuracy:** ART has demonstrated substantial improvements over traditional few-shot prompting and automatic CoT methods. It excels particularly on unseen tasks, as evidenced by its strong performance on benchmarks like BigBench and MMLU.
  2. **Enhanced flexibility:** By allowing for human intervention to update reasoning steps or tools, ART offers a robust framework that can adapt to new information and requirements, improving overall performance.
  3. **Efficient tool integration:** ART’s automated handling of tool integration and intermediate reasoning steps reduces the need for manually scripted interactions, streamlining the process and increasing efficiency.



The Automatic Reasoning and Tool-Use (ART) framework represents a significant advancement in the integration of reasoning and tool usage within LLMs. By automating the generation of intermediate steps and dynamically incorporating tool outputs, ART enhances the model’s ability to handle complex, multi-step tasks. Its zero-shot generalization capabilities and extensibility make it a powerful tool for improving performance on a wide range of tasks, surpassing traditional prompting methods and showcasing the potential for future advancements in automated reasoning.

### Automatic Prompt Engineer (APE)

In the realm of prompt engineering for large language models (LLMs), optimizing the prompts to enhance model performance is a critical challenge. [Zhou et al. (2022)](https://arxiv.org/abs/2211.01910) introduced the Automatic Prompt Engineer (APE) framework to tackle this challenge by automating the generation and selection of prompts. This innovative approach frames prompt optimization as a black-box optimization problem, leveraging LLMs to generate and evaluate instruction candidates.

**Overview of Automatic Prompt Engineer (APE):**

APE is designed to streamline the process of creating effective prompts by automating both the generation of potential prompts and the selection of the best-performing ones. The framework aims to improve the efficiency and effectiveness of prompts beyond what is achievable through manual engineering alone.

**How APE works?**

  1. **Instruction generation:** The first phase involves using a large language model as an inference engine to generate a set of candidate instructions or prompts for a given task. This model is provided with output demonstrations that guide the generation process, ensuring that the prompts are relevant and tailored to the task at hand.
  2. **Execution and evaluation:** Once the candidate instructions are generated, they are executed using a target model. The performance of each instruction is assessed based on computed evaluation scores. These scores help determine which instruction is most effective for the given task.
  3. **Selection of optimal prompts:** The prompt with the highest evaluation score is selected as the most appropriate for the task. This selection process is automated, allowing for a more efficient and objective determination of the best prompt.



**Key findings and contributions:**

  * **Improved prompt performance:** APE has demonstrated the capability to discover more effective zero-shot chain-of-thought (CoT) prompts compared to manually engineered prompts. For instance, APE identified a prompt that outperforms the commonly used "Let's think step by step" prompt proposed by [Kojima et al. (2022)](https://arxiv.org/abs/2205.11916).
  * **Black-Box Optimization approach:** By framing prompt generation as a black-box optimization problem, APE allows for a systematic exploration of potential prompts without requiring manual intervention. This approach enhances the ability to identify high-performing prompts that might not be obvious through traditional methods.



**Related research:**

  * **Prompt-OIRL:** Introduces offline inverse reinforcement learning to generate query-dependent prompts, optimizing prompt performance based on task-specific queries.
  * **OPRO:** Demonstrates the use of LLMs to optimize prompts with the "Take a deep breath" technique, which has been shown to improve performance on mathematical problems.
  * **AutoPrompt:** Proposes a gradient-guided search approach to automatically create prompts for a variety of tasks, focusing on optimizing prompt effectiveness.
  * **Prefix tuning:** Offers a lightweight alternative to fine-tuning by appending a trainable continuous prefix to the model for natural language generation tasks.
  * **Prompt tuning:** Introduces a method for learning soft prompts through backpropagation, allowing for more flexible and adaptable prompt optimization.



The Automatic Prompt Engineer (APE) framework represents a significant advancement in prompt optimization by automating the generation and evaluation of prompts. Through its black-box optimization approach, APE identifies more effective prompts than traditional methods, enhancing the performance of LLMs on various tasks. This framework, along with related research, highlights the evolving landscape of prompt engineering and its potential to drive further improvements in language model applications.

### Active-prompt

In the field of large language model (LLM) prompting, chain-of-thought (CoT) methods often rely on a static set of human-annotated examples. While effective, this approach may not always provide the most suitable examples for diverse tasks, potentially limiting performance. To address this limitation, [Diao et al. (2023)](https://arxiv.org/pdf/2302.12246) introduced a novel prompting technique known as Active-Prompt, designed to dynamically adapt LLMs to task-specific prompts by leveraging an iterative example refinement process.

**Overview of Active-prompt:**

Active-Prompt enhances the adaptability of LLMs by refining and selecting task-specific examples through an active learning process. This approach aims to continuously improve the quality of the prompts used by incorporating feedback from human annotators, thus optimizing the CoT reasoning process for various tasks.

**How active-prompt works?**

  1. **Initial query and example generation:** The process begins by querying the LLM with or without a few initial CoT examples. These examples serve as a starting point for generating a range of possible answers to a set of training questions. The LLM generates k candidate answers for each question, providing a broad perspective on potential responses.
  2. **Uncertainty measurement:** An uncertainty metric is then calculated based on the k answers generated. This metric assesses the level of disagreement among the candidate answers, identifying questions where the model's responses exhibit significant variance. High uncertainty indicates areas where the model may need additional clarification or refinement.
  3. **Human annotation:** The most uncertain questions, as determined by the uncertainty metric, are selected for human annotation. Human experts review these questions and provide additional examples or corrections, enhancing the quality and relevance of the prompts.
  4. **Incorporation of annotated examples:** The newly annotated examples are integrated into the existing prompt set. These updated examples are then used to infer responses for each question, helping to improve the model’s performance and accuracy in handling task-specific queries.



**Key contributions and benefits:**

  * **Dynamic example refinement:** Active-Prompt introduces a dynamic approach to refining CoT examples, addressing the limitations of static example sets. By continuously updating the prompt examples based on uncertainty, the technique ensures that the model adapts to varying task requirements.
  * **Enhanced adaptability:** The iterative nature of Active-Prompt allows LLMs to better handle a diverse range of tasks by incorporating human feedback into the prompting process. This adaptability leads to more effective and contextually relevant responses.
  * **Improved performance:** By focusing on areas of high uncertainty and refining examples accordingly, Active-Prompt improves the overall performance of LLMs on specific tasks. This results in more accurate and reliable outputs, particularly for complex or nuanced queries.



**Related research:**

  * **CoT methods:** Traditional chain-of-thought methods rely on fixed sets of human-annotated examples, which may not always be optimal for all tasks.
  * **Dynamic prompting techniques:** Active-Prompt builds on the concept of dynamic prompting by incorporating active learning and iterative refinement, enhancing the effectiveness of LLMs for diverse applications.



Active-Prompt offers a significant advancement in the realm of LLM prompting by introducing a dynamic and adaptive approach to example selection and refinement. By leveraging uncertainty metrics and human annotation, this technique optimizes the CoT reasoning process, ensuring that LLMs are better equipped to handle a wide range of task-specific queries. This approach represents a valuable addition to the toolkit of prompt engineering methods, promoting improved performance and adaptability in language model applications.

## Prompt engineering: The step-by-step process

Prompt engineering is a multi-step process that involves several key tasks. Here they are:

### Understanding the problem

Understanding the problem is a critical first step in prompt engineering. It requires not just knowing what you want your model to do, but also understanding the underlying structure and nuances of the task at hand. This is where the art and science of problem analysis in the context of AI comes into play.

The type of problem you are dealing with greatly influences the approach you will take when crafting prompts. For instance:

  * **Question-answering tasks** : For a question-answering task, you would need to understand the type of information needed in the answer. Is it factual? Analytical? Subjective? Also, you would have to consider whether the answer requires reasoning or context.
  * **Text generation tasks** : If it is a text generation task, factors like the desired length of the output, its format (story, poem, article), and its tone or style come into play.
  * **Sentiment analysis tasks** : For sentiment analysis, the prompt should be structured to guide the model to recognize subjective expressions and discern the sentiment from the text.



Understanding the problem also involves identifying any potential challenges or limitations associated with the task. For instance, a task might involve domain-specific language, slang, or cultural references, which the model may or may not be familiar with.

Moreover, understanding the problem thoroughly helps in anticipating how the model might react to different prompts. You might need to provide explicit instructions, or use a specific format for the prompt. Or, you may need to iterate and refine the prompts several times to get the desired output.

Ultimately, a deep understanding of the problem allows for the creation of more effective and precise prompts, which in turn leads to better performance from the large language model.

### Crafting the initial prompt

Crafting the initial prompt is an essential task in the process of prompt engineering. This step involves the careful composition of an initial set of instructions to guide the language model's output, based on the understanding gained from the problem analysis.

The main objective of a prompt is to provide clear, concise, and unambiguous directives to the language model. It acts as a steering wheel, directing the model to the required path and desired output. A well-structured prompt can effectively utilize the capabilities of the model, producing high-quality and task-specific responses.

In some scenarios, especially in tasks that require a specific format or context-dependent results, the initial prompt may also incorporate a few examples of the desired inputs and outputs, known as few-shot examples. This method is often used to give the model a clearer understanding of the expected result.

For instance, if you want the model to translate English text into French, your prompt might include a few examples of English sentences and their corresponding French translations. This helps the model to grasp the pattern and the context better.

Remember, while crafting the initial prompt, it is also essential to maintain flexibility. The ideal output is seldom achieved with the first prompt attempt. Often, you would need to iterate and refine the prompts, based on the model's responses, to achieve the desired results. This process of iterative refinement is an integral part of prompt engineering.

### Evaluating the model's response

Evaluating the model's response is a crucial phase in prompt engineering that follows after the initial prompt has been utilized to generate a model response. This step is key in understanding the effectiveness of the crafted prompt and the language model's interpretive capacity.

The first thing to assess is whether the model's output aligns with the task's intended goal. For example, if the task is about translating English sentences into Spanish, does the output correctly and accurately render the meaning in Spanish? Or if the task is to generate a summary of a lengthy article, does the output present a concise and coherent overview of the article's content?

When the model's response does not meet the desired objective, it's essential to identify the areas of discrepancy. This could be in terms of relevance, accuracy, completeness, or contextual understanding. For instance, the model might produce a grammatically correct sentence that is contextually incorrect or irrelevant.

Upon identifying the gaps, the aim should be to understand why the model is producing such output. Is the prompt not explicit enough? Or is the task too complex for the model's existing capabilities? Answering these questions can provide insights into the limitations of the model as well as the prompt, guiding the next step in the prompt engineering process - Refining the prompts.

Evaluating the model's response is a crucial iterative process in prompt engineering, acting as a feedback loop that consistently informs and improves the process of crafting more effective prompts.

### Iterating and refining the prompt

Iterating and refining the prompt is an essential step in prompt engineering that arises from the evaluations of the model's response. This stage centers on improving the effectiveness of the prompt based on the identified shortcomings or flaws in the model's output.

When refining a prompt, several strategies can be employed. These strategies are predominantly influenced by the nature of the misalignment between the model's output and the desired objective.

For instance, if the model's response deviates from the task's goal due to a lack of explicit instructions in the prompt, the refinement process may involve making the instructions clearer and more specific. Explicit instructions help ensure that the model comprehends the intended objective and doesn't deviate into unrelated content or produce irrelevant responses.

On the other hand, if the model is struggling to understand the structure of the task or the required output, it may be beneficial to provide more examples within the prompt. These examples can act as guidelines, demonstrating the correct form and substance of the desired output.

Similarly, the format or structure of the prompt itself can be altered in the refinement process. The alterations could range from changing the order of sentences or the phrasing of questions to the inclusion of specific keywords or format cues.

The iteration and refinement process in prompt engineering is cyclic, with multiple rounds of refinements often necessary to arrive at a prompt that most effectively elicits the desired output from the model. It is a process that underlines the essence of prompt engineering - the fine-tuning of language to communicate effectively with large language models.

### Testing the prompt on different models

Testing the prompt on different models is a significant step in prompt engineering that can provide in-depth insights into the robustness and generalizability of the refined prompt. This step entails applying your prompt to a variety of [large language models](https://www.leewayhertz.com/how-to-train-domain-specific-llms/) and observing their responses. It is essential to understand that while a prompt may work effectively with one model, it may not yield the desired result when applied to another. This is because different models may have different architectures, training methodologies, or datasets that influence their understanding and response to a particular prompt.

The size of the model plays a significant role in its ability to understand and respond accurately to a prompt. For instance, larger models often have a broader context window and can generate more nuanced responses. On the other hand, smaller models may require more explicit prompting due to their reduced contextual understanding.

The model's architecture, such as [transformer-based models](https://www.leewayhertz.com/transformer-model-development-services/) like GPT-3 or LSTM-based models, can also influence how it processes and responds to prompts. Some architectures may excel at certain tasks, while others may struggle, and this can be unveiled during this testing phase.

Lastly, the training data of the models plays a crucial role in their performance. A model trained on a wide range of topics and genres may provide a more versatile response than a model trained on a narrow, specialized dataset.

By testing your prompt across various models, you can gain insights into the robustness of your prompt, understand how different model characteristics influence the response, and further refine your prompt if necessary. This process ultimately ensures that your prompt is as effective and versatile as possible, reinforcing the applicability of prompt engineering across different large language models.

### Scaling the prompt

After refining and testing your prompt to a point where it consistently produces desirable results, it's time to scale it. Scaling, in the context of prompt engineering, involves extending the utility of a successfully implemented prompt across broader contexts, tasks, or automation levels.

  1. Automating prompt generation: Depending on the nature of the task and the model's requirements, it may be possible to automate the process of generating prompts. This could involve creating a script or a tool that generates prompts based on certain parameters or rules. Automating prompt generation can save a significant amount of time, especially when dealing with a high volume of tasks or data. It can also reduce the chance of human error and ensure consistency in the prompt generation process.
  2. Creating variations of the prompt: Another way to scale a prompt is to create variations that can be used for related tasks. For example, if you have a prompt that successfully guides a model in performing sentiment analysis on product reviews, you might create variations of this prompt to apply it to movie reviews, book reviews, or restaurant reviews. This approach leverages the foundational work that went into creating the original prompt and allows you to address a wider range of tasks more quickly and efficiently.



Scaling the prompt is the final step in the prompt engineering process, reflecting the successful development of an effective prompt. It represents a transition from development to deployment, as the prompt begins to be used in real-world applications on a broader scale.

It's worth noting that prompt engineering is an iterative process. It requires ongoing testing and refinement to optimize the model's performance for the given task.

## Key elements of a prompt

Delving into the world of prompt engineering, we encounter four pivotal components that together form the cornerstone of this discipline. These are instructions, context, input data, and output indicators. Together, they provide a framework for effective communication with large language models, shaping their responses and guiding their operations. Here, we explore each of these elements in depth, helping you comprehend and apply them efficiently in your [AI development](https://www.leewayhertz.com/artificial-intelligence-application-development-company/) journey.

  * **Instruction** : This is the directive given to the model that details what is expected in terms of the task to be performed. This could range from "translate the following text into French" to "generate a list of ideas for a science fiction story". The instruction is usually the first part of the prompt and sets the overall task for the model.
  * **Context** : This element provides additional information that can guide the model's response. For instance, in a translation task, you might provide some background on the text to be translated (like it's a dialogue from a film or a passage from a scientific paper). The context can help the model understand the style, tone, and specifics of the information needed.
  * **Input data** : This refers to the actual data that the model will be working with. In a translation task, this would be the text to be translated. In a question-answering task, this would be the question being asked.
  * **Output indicator** : This part of the prompt signals to the model the format in which the output should be generated. For instance, you might specify that you want the model's response in the form of a list, a paragraph, a single sentence, or any other specific structure. This can help narrow down the model's output and guide it towards more useful responses.



While these elements are not always required in every prompt, a well-crafted prompt often includes a blend of these components, tailored to the specific task at hand. Each element contributes to shaping the model's output, guiding it towards generating responses that align with the desired goal.

## Contact LeewayHertz’s prompt engineers today!

Enhance the performance of your AI models with our prompt engineering services

[Learn More](https://www.leewayhertz.com/hire-prompt-engineers/?utm_source=prompt-engineering-Insight&utm_medium=CTA)

## How to design prompts?

### Importance of LLM settings

Designing prompts for a large language model involves understanding and manipulating specific settings that can steer the model's output. These settings can be modified either directly or via an API.

Key settings include the 'Temperature' and 'Top_p' parameters. The 'Temperature' parameter controls the randomness of the model's output. Lower values make the model's output more deterministic, favoring the most probable next token. This is useful for tasks requiring precise and factual answers, like a fact-based question-answer system. On the other hand, increasing the 'Temperature' value induces more randomness in the model's responses, allowing for more creative and diverse results. This is beneficial for creative tasks like poem generation.

The 'Top_p' parameter, used in a sampling technique known as nucleus sampling, also influences the determinism of the model's response. A lower 'Top_p' value results in more exact and factual answers, while a higher value increases the diversity of the responses.

One key recommendation is to adjust either 'Temperature' or 'Top_p,' but not both simultaneously, to prevent overcomplicating the system and to better control the effect of these settings.

Remember that the performance of your prompt may vary depending on the version of LLM you are using, and it's always beneficial to iterate and experiment with your settings and prompt design.

### Key strategies for successful prompt design

Here are some tips to keep in mind while you are designing your prompts

#### Begin with the basics

While embarking on the journey of designing prompts you need to remember that it's a step-by-step process that demands persistent tweaking and testing to achieve excellence. Platforms like OpenAI or Cohere provide a user-friendly environment for this venture. Kick off with basic prompts, gradually enriching them with more components and context as you strive for enhanced outcomes. Maintaining different versions of your prompts is crucial in this progression. Through this guide, you will discover that clarity, simplicity, and precision often lead to superior results.

For complex tasks involving numerous subtasks, consider deconstructing them into simpler components, progressively developing as you achieve promising results. This approach prevents an overwhelming start to the prompt design process.

#### Crafting effective prompts: The power of instructions

As a prompt designer, one of your most potent tools is the instruction you give to the language model. Instructions such as "Write," "Classify," "Summarize," "Translate," "Order," etc., guide the model to execute a variety of tasks.

Remember, crafting an effective instruction often involves a considerable amount of experimentation. To optimize the instruction for your specific use case, test different instruction patterns with varying keywords, contexts, and data types. The rule of thumb here is to ensure the context is as specific and relevant to your task as possible.

Here is a practical tip: most prompt designers suggest placing the instruction at the start of the prompt. A clear separator, like "###", could be used to distinguish the instruction from the context. For example:

"### Instruction ### Translate the following text to French:

Text: "Good morning!" By following these guidelines, you will be well on your way to creating effective and precise prompts.

#### The essence of specificity in prompt design

In the realm of prompt design, specificity is vital. The more accurately you define the task and instruction, the more aligned the outcomes will be with your expectations. It's not so much about using certain tokens or keywords, but rather about formulating a well-structured and descriptive prompt.

A useful technique is to include examples within your prompts; they can guide the model to produce the output in the desired format. For instance, if you are seeking a summarization of a text in three sentences, your instruction could be:

"Summarize the following text into 3 sentences: …"

Keep in mind that while specificity is important, there is a balance to be found. You should be conscious of the prompt's length, as there are limitations to consider. Additionally, overloading the prompt with irrelevant details may confuse the model rather than guiding it. The goal is to include details that meaningfully contribute to the task at hand.

Prompt design is a process of constant experimentation and iteration. Always seek to refine and enhance your prompts for optimal outcomes. Experiment with different levels of specificity and detail to find what works best for your unique applications.

#### Sidestepping ambiguity in prompt design

While prompt design requires a balance of detail and creativity, it is crucial to avoid ambiguity or impreciseness. Much like clear communication, precise instructions yield better results. An overly clever or convoluted prompt can lead to less desirable outcomes. Instead, focus on clarity and specificity.

For instance, let's say you want your model to generate a brief definition of the term 'Artificial Intelligence'. An imprecise prompt might be:

"Talk about this thing that's being used a lot these days, Artificial Intelligence."

While the model may understand this prompt, it's indirect and lacks clarity. You may receive a lengthy discourse rather than the succinct definition you desire. A clearer, more direct prompt could be:

"Define the term 'Artificial Intelligence' in one sentence."

This prompt is precise and directs the model to generate a specific output. The output, in this case, could be:

"Artificial Intelligence is a branch of computer science focused on creating machines capable of mimicking human intelligence."

Through avoiding ambiguity in your prompts, you can effectively guide the model to produce the desired output.

#### Choosing clarity over restrictions

In prompt designing, it's often more beneficial to instruct the model on what to do, rather than dictating what not to do. This approach promotes precision and directs the model towards generating useful responses.

Consider, for instance, you are trying to get a language model to recommend a book. An ineffective instruction might be:

"Do not recommend any books that are not in the top 10 best sellers list."

This statement might confuse the model as it's primarily focusing on what not to do. A more productive instruction would be:

"Recommend a book from the top 10 best sellers list."

This is direct, clear, and focuses the model on the task you desire. The output could be:

"One of the top 10 best sellers right now is 'The Code Breaker' by Walter Isaacson. It's a fascinating look into the world of genetics and biochemistry."

Emphasizing the desired action in your prompt, rather than the prohibited ones, ensures the model clearly understands your expectations and is more likely to deliver an appropriate response.

## A case study on importance of effective prompt

To illustrate the significance of a carefully composed prompt, let's say we are developing an XGBoost model and our goal is to author a Python script that carries out hyperparameter optimization. The data we are working with is voluminous and not evenly distributed.

We are going to experiment with two distinct prompts both intended to generate code aiding hyperparameter optimization. The first prompt offers just the basic context, whereas the second one is enhanced with some supplementary directives.

**Prompt 1: Write Python code that executes hyperparameter tuning on an XGBoost classifier.**

Here is the code that ChatGPT generates (model used here in GPT4):
    
    
    import pandas as pd
    from sklearn.model_selection import train_test_split, GridSearchCV
    from xgboost import XGBClassifier
     
    # Load dataset
    data = pd.read_csv('PATH_TO_YOUR_DATA')
     
    # Assume 'target' is your target variable
    X = data.drop('target', axis=1)
    y = data['target']
     
    # Split dataset into training set and test set
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3) # 70% training and 30% test
     
    # Define XGBoost classifier
    xgb = XGBClassifier()
     
    # Define hyperparameter grid
    param_grid = {
        'n_estimators': [100, 200, 500],
        'learning_rate': [0.01, 0.05, 0.1],
        'max_depth': [3, 4, 5],
        'colsample_bytree': [0.3, 0.7]
    }
     
    # Initialize GridSearchCV
    grid = GridSearchCV(xgb, param_grid, cv=5)
     
    # Fit GridSearchCV
    grid.fit(X_train, y_train)
     
    # Print best parameters
    print("Best Parameters: ", grid.best_params_)
     
    # Print best score
    print("Best Score: ", grid.best_score_)
    
    

The code uses Scikit-learn's GridSearchCV for hyperparameter tuning in an XGBoost classifier. Please replace "PATH_TO_YOUR_DATA" with the actual path to your dataset and ensure that your target variable is properly defined. Also, adjust the parameters as per your requirements.

The solution provided does function as expected, but it may not perform optimally for larger datasets or those with imbalanced classes. The grid search approach, while thorough, can be both inefficient and time-consuming. Moreover, using accuracy as a metric can be misleading when dealing with imbalanced data, often giving a false sense of model performance.

In essence, this underlines how a lack of sufficient information in a prompt can lead to less-than-ideal solutions.

Now, let's enhance our prompt by incorporating additional instructions and observe how it impacts the resulting output.

**Prompt 2: Write Python code that executes hyperparameter tuning on an XGBoost classifier. Carry out 50 trials using the Optuna library, with f-1 score as the evaluation metric. Only consider the following hyperparameters: n_estimators, learning_rate, gamma, and max_depth.**

Here is the generated code
    
    
    import optuna
    from sklearn.metrics import f1_score
    from sklearn.model_selection import train_test_split
    from xgboost import XGBClassifier
     
    # Assuming X and y are your features and target
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
     
    def objective(trial):
        n_estimators = trial.suggest_int("n_estimators", 2, 150)
        learning_rate = trial.suggest_loguniform("learning_rate", 1e-5, 1e-2)
        gamma = trial.suggest_loguniform("gamma", 1e-8, 1.0)
        max_depth = trial.suggest_int("max_depth", 1, 9)
         
        classifier_obj = XGBClassifier(
            n_estimators=n_estimators,
            learning_rate=learning_rate,
            gamma=gamma,
            max_depth=max_depth,
            objective="binary:logistic")
         
        classifier_obj.fit(X_train, y_train)
         
        y_pred = classifier_obj.predict(X_test)
        return f1_score(y_test, y_pred)
     
    study = optuna.create_study(direction="maximize")
    study.optimize(objective, n_trials=50)
     
    trial = study.best_trial
     
    print(f"Best Trial: score {trial.value}, params {trial.params}")
    
    

In this revised prompt, there's less ambiguity, with clear indications about the library to use, which hyperparameters to adjust, how many trials to conduct, and the evaluation metric to employ. Just two extra sentences compared to the original prompt, yet it delivers a significantly better code output!

As observed, the code generated by ChatGPT makes use of the Optuna library for Bayesian search on the specified four hyperparameters, utilizing the f1-score as the evaluation measure. This approach is far more efficient and less time-intensive than the one proposed in response to the earlier prompt.

## Prompt engineering best practices

### Craft detailed and direct instructions

  * _Strategy 1:_ Use delimiters such as `, “““, < >, <tag> </tag> to distinguish different sections of the input. This helps in structuring your input effectively and preventing prompt errors. For instance, using the `delimiters to specify the text to be summarized.
  * _Strategy 2:_ Request for a structured output. This could be in a JSON format, which can easily be converted into a list or dictionary in Python later on.
  * _Strategy 3:_ Confirm whether conditions are met. The prompt can be designed to verify assumptions first. This is particularly helpful when dealing with edge cases. For example, if the input text doesn't contain any instructions, you can instruct the model to write "No steps provided".
  * _Strategy 4:_ Leverage few-shot prompting. Provide the model with successful examples of completed tasks, then ask the model to carry out a similar task.



### Allow the model time to 'Think'

  * _Strategy 1:_ Detail the steps needed to complete a task and demand output in a specified format. For complex tasks, breaking them down into smaller steps can be beneficial, just as humans often find step-by-step instructions helpful. You can ask the model to follow a logical sequence or chain of reasoning before arriving at the final answer.
  * _Strategy 2:_ Instruct the model to work out its solution before jumping to a conclusion. This helps the model in thoroughly processing the task at hand before delivering the output.



### Opt for the latest model

To attain optimal results, it is advisable to use the most advanced models.

### Provide detailed descriptions

Clarity is crucial. Be specific and descriptive about the required context, outcome, length, format, style, etc. For instance, instead of simply requesting a poem about OpenAI, specify details like poem length, style, and a particular theme, such as a recent product launch.

### Use examples to illustrate desired output format

The model responds better to specific format requirements shown through examples. This approach also simplifies the process of parsing multiple outputs programmatically.

### Start with zero-shot, then few-shot, and finally fine-tune

For complex tasks, start with zero-shot, then proceed with few-shot techniques. If these methods don't yield satisfactory results, consider fine-tuning the model.

### Eliminate vague and unnecessary descriptions

Precision is essential. Avoid vague and "fluffy" descriptions. For instance, instead of saying, "The description should be fairly short," provide a clear guideline such as, "Use a 3 to 5 sentence paragraph to describe this product."

### Give direct instructions over prohibitions

Instead of telling the model what not to do, instruct it on what to do. For instance, in a customer service conversation scenario, instruct the model to diagnose the problem and suggest a solution, avoiding any questions related to personally identifiable information (PII).

### Use leading words for code generation

For code generation tasks, nudge the model towards a particular pattern using leading words. This might include using words like 'import' to hint the model that it should start writing in Python, or 'SELECT' for initiating a SQL statement.

## Contact LeewayHertz’s prompt engineers today!

Enhance the performance of your AI models with our prompt engineering services

[Learn More](https://www.leewayhertz.com/hire-prompt-engineers/?utm_source=prompt-engineering-Insight&utm_medium=CTA)

## Applications of prompt engineering

### Program-aided Language Model (PAL)

Program-aided language models in prompt engineering involve integrating programmatic instructions and structures to enhance the capabilities of language models. By incorporating additional programming logic and constraints, PAL enables more precise and context-aware responses. This approach allows developers to guide the model's behavior, specify the desired output format, provide relevant examples, and refine prompts based on intermediate results. By leveraging programmatic guidance, PAL techniques empower language models to generate more accurate and tailored responses, making them valuable tools for a wide range of applications in natural language processing.

Here is an example of how PAL can be applied in prompt engineering:

Prompt:
    
    
    Given a list of numbers, compute the sum of all even numbers.
    Input: [2, 5, 8, 10, 3, 6]
    Output: The sum of all even numbers is 26.

In this example, the prompt includes a programmatic instruction to compute the sum of even numbers in a given list. By providing this specific task and format, the language model guided by PAL techniques can generate a response that precisely fulfills the desired computation. The integration of programmatic logic and instructions in the prompt ensures accurate and contextually appropriate results.

### Generating data

Generating data is an important application of prompt engineering with large language models (LLMs). LLMs have the ability to generate coherent and contextually relevant text, which can be leveraged to create synthetic data for various purposes.

For example, in natural language processing tasks, generating data using LLMs can be valuable for training and evaluating models. By designing prompts that instruct the LLM to generate specific types of data, such as question-answer pairs, text summaries, or dialogue interactions, researchers and practitioners can create large volumes of labeled training data. This synthetic data can then be used to train and improve NLP models, as well as to evaluate their performance.

Here is an example:

Prompt:
    
    
    Generate 100 question-answer pairs about famous landmarks.

Using this prompt, the LLM can generate a diverse set of question-answer pairs related to famous landmarks around the world. The generated data can be used to enhance question-answering models or to augment existing datasets for training and evaluation.

By employing prompt engineering techniques, researchers and developers can effectively utilize LLMs to generate data that aligns with their specific needs, enabling them to conduct experiments, evaluate models, and advance various domains of research.

### Generating code

Generating code is another application of prompt engineering with large language models. LLMs can be prompted to generate code snippets, functions, or even entire programs, which can be valuable in [software development](https://www.leewayhertz.com/software-development-company/), automation, and programming education.

For example, let's consider a scenario where a developer wants to generate a Python function that calculates the factorial of a number:

Prompt:
    
    
    Write a Python function named "factorial" that takes an integer as input and returns its factorial.

By providing this specific prompt to the LLM, it can generate code that implements the factorial function in Python:

Generated Code:
    
    
    def factorial(n):
        if n == 0 or n == 1:
            return 1
        else:
            return n * factorial(n - 1)

The generated code demonstrates the recursive implementation of the factorial function in Python.

Prompt engineering allows developers to design prompts with clear instructions and specifications, such as function names, input requirements, and desired output formats. By carefully crafting prompts, LLMs can be guided to generate code snippets tailored to specific programming tasks or requirements.

This application of prompt engineering can be highly beneficial for developers seeking assistance in code generation, automating repetitive tasks, or even for educational purposes where learners can explore different code patterns and learn from the generated examples.

### Function calling with LLMs

Function calling is a sophisticated technique that enhances the interaction between large language models (LLMs) and external tools or APIs. It allows LLMs like GPT-4 and GPT-3.5 to effectively use external functions by converting natural language queries into structured API calls, which are executed to retrieve or process information.  
Overview

**Function calling involves:**

**Generating function calls:** LLMs identify when a function call is required and produce a JSON structure detailing the function's arguments.  
**Executing functions:** Functions act as external tools within the AI application, and multiple functions can be invoked within a single request.  
**Integrating outputs:** The LLM combines the results from function calls with its own responses to provide comprehensive answers.

**Applications and use cases**

**Conversational agents:** Function calling enables chatbots to handle complex queries by interfacing with external APIs. For example, a query like "What is the weather like in London?" can be converted into a function call to retrieve current weather data, which is then integrated into the chatbot's response.

**Natural Language to API calls:** This technique transforms natural language instructions into structured API calls or database queries, facilitating seamless interaction with data sources. This capability is useful for creating applications that need to convert user queries into actionable API requests.

**Data extraction and tagging:** LLMs use function calling to extract specific information from documents or databases, such as identifying and tagging entities within a text. This can be applied in scenarios like named entity recognition and sentiment analysis.

**Knowledge retrieval engines:** Function calling helps in developing systems that access and utilize knowledge bases to answer user queries effectively. This application is particularly valuable for creating conversational knowledge retrieval systems that need to provide accurate and contextually relevant responses.

**Mathematical problem solving:** Custom functions can be defined for solving complex mathematical problems, which require multiple steps or advanced calculations. Function calling allows LLMs to perform these calculations by invoking the necessary functions.

**API integration:** This capability is essential for integrating LLMs with various APIs to fetch data or perform actions based on user input. For instance, a LLM could convert a natural language query into a valid API call to retrieve information from an external source.

**Information extraction:** Function calling is effective for extracting specific details from given inputs, such as retrieving relevant news stories or references from an article. This application supports tasks like data retrieval and summarization.

**Example implementation with GPT-4**

To illustrate how function calling works with GPT-4, let’s consider a scenario where a user wants to know the current weather in London. Here’s how the process unfolds:

  * **Define the function:** Start by setting up a function that will be used to fetch weather information. This function, named get_current_weather, is designed to take two pieces of information: the location (such as London) and the unit of temperature (celsius or fahrenheit). The function doesn’t execute on its own but rather generates a set of instructions, known as JSON, detailing what information is needed to call it.
  * **Generate the request:** When the user asks the question, "What is the weather like in London?", GPT-4 processes this request and determines that it needs to call the get_current_weather function. It creates a structured request containing the location and the temperature unit specified by the user. This structured request is essentially a description of what the function needs to do but does not execute the function directly.
  * **Process the function call:** The structured request is then used to query an external weather service. The system uses the function’s parameters, which GPT-4 provided, to call the weather API and retrieve the current weather data for London.
  * **Integrate and respond:** Once the weather data is obtained from the external service, it is fed back into the system. GPT-4 then uses this information to generate a final, comprehensive response to the user's original question. This response combines the retrieved data with GPT-4’s own capabilities to provide an accurate and informative answer about the weather in London.



In summary, GPT-4 enhances its functionality by converting natural language queries into structured requests that interact with external tools, such as APIs, to deliver precise and relevant information to users.

### Context caching with Gemini 1.5 Flash

Google’s recent release, the Gemini 1.5 Flash model, introduces a powerful feature known as context-caching. This feature, available through the Gemini APIs, enhances the efficiency of querying large volumes of text by maintaining relevant context over time. This guide illustrates how to leverage context-caching with Gemini 1.5 Flash to analyze a substantial dataset of machine learning (ML) papers accumulated over the past year.

**Use case: Analyzing a year’s worth of ML papers**

In this example, context caching is used to manage and query a comprehensive collection of [ML paper summaries stored in a text file](https://github.com/dair-ai/ML-Papers-of-the-Week). The primary goal is to streamline the analysis of these documents by efficiently handling queries about the content.

**Implementation steps:**

  1. **Data preparation:** Begin by converting the document containing the ML paper summaries into a plain text file. This file will serve as the input for the context-caching process.
  2. **Utilizing the Gemini API:** Upload the prepared text file using the Google Generative AI library. This step involves integrating the text data with the Gemini API.
  3. **Implementing context caching:** Create a context cache using the caching.CachedContent.create() function, specifying the following details: 
     * **Model specification** : Indicate the use of the Gemini 1.5 Flash model.
     * **Cache naming:** Assign a name to the cache for identification.
     * **Instruction definition:** Provide an instruction to guide the model's responses (e.g., “You are an expert in AI research…”).
     * **Time-to-Live (TTL):** Set a TTL for the cache, such as 15 minutes, to control how long the cache remains valid.
  4. **Creating the model:** Instantiate a generative model instance that uses the cached content for efficient querying.
  5. **Querying the model:** Start querying the model with natural language questions. Examples include: 
     * “What are the latest AI papers from this week?”
     * “Can you list papers that mention Mamba, including their titles and summaries?”
     * “What innovations are discussed in long-context LLMs? Provide the titles and summaries of relevant papers.”



**Results and benefits:**

The implementation of context-caching demonstrated significant improvements in efficiency. The model successfully retrieved and summarized data from the text file without needing to resend the entire file with each query. This approach offers several advantages:

  * **Rapid data analysis:** Researchers can quickly analyze extensive datasets without manual searches.
  * **Targeted retrieval:** Specific findings can be obtained efficiently, reducing the time spent locating information.
  * **Interactive research:** Enables dynamic and interactive research sessions, optimizing the use of prompt tokens.



Context-caching with Gemini 1.5 Flash proves to be a valuable tool for handling large volumes of research data, enhancing the overall effectiveness of querying and analysis processes.

### Generating synthetic datasets for RAG

In machine learning, acquiring labeled data often presents a major challenge. Traditional methods often involve extensive data collection and labeling phases, which can delay the development of AI solutions by months. However, advancements in large language models (LLMs) have introduced new possibilities. These models can now generate synthetic data, which accelerates the development process and enhances model training, particularly for Retrieval Augmented Generation (RAG) tasks.

**Understanding Retrieval Augmented Generation (RAG)**

Retrieval Augmented Generation (RAG) is a method designed for knowledge-intensive tasks where relying solely on a model’s pre-existing knowledge may be insufficient. RAG combines a retrieval model with a text generator to provide more accurate and contextually relevant responses. The retrieval component identifies pertinent documents, which are then processed by the generative model to produce answers or summaries.

Despite its utility, the performance of the retrieval model can vary, especially in different languages or specialized domains. For instance, a RAG system tasked with handling Czech legal documents or Indian tax regulations might struggle with document retrieval if the model is not adequately trained.

**Leveraging synthetic data for enhanced performance**

One innovative approach to address the limitations of retrieval models is to use existing LLMs to generate synthetic data. This technique involves creating a dataset through prompt-based queries, which can then be used to train or fine-tune retrieval models. This method helps in overcoming data scarcity issues and improves performance in specific domains or languages.

**Steps for generating synthetic data:**

**Data preparation:** Begin by defining a few manually labeled examples related to your specific retrieval task. For instance, if the task is to find counter-arguments to legal arguments, prepare a set of example arguments and their corresponding counter-argument queries.

**Generating synthetic data:** Use LLMs like ChatGPT or GPT-4 to generate additional query-document pairs. For each example, provide a prompt that instructs the model to create a new query for a given document. The synthetic data generated from these prompts can be used to augment the training dataset.  
**Example prompt:** “Task: Identify a counter-argument for the given argument. Argument #1: {insert passage here}. A concise counter-argument query related to the argument #1: {insert query here}.”

**Creating the model:** Once the synthetic data is generated, integrate it with your existing dataset to train or fine-tune the retrieval model. This model will then be better equipped to handle specific queries related to your domain.

**Evaluating and refining:** Assess the performance of the retrieval model with the new synthetic data. If needed, refine the synthetic data generation process by adjusting prompts or including more varied examples to improve model accuracy.

**Advantages of synthetic data generation:**

**Cost-efficiency:** Generating synthetic data is often more cost-effective than manual data labeling. For instance, generating a dataset of 50,000 documents using LLMs can be significantly cheaper than gathering and annotating the same amount of real data.

**Speed:** Synthetic data allows for rapid iteration and model training, reducing the time required to develop and test new AI features.

**Domain specialization:** Synthetic data can be tailored to specific domains or languages, improving the model's performance in areas where real data is scarce.

In summary, using LLMs to generate synthetic datasets for RAG can significantly enhance the development and performance of retrieval models. This approach not only alleviates the challenges associated with data scarcity but also accelerates the training process, providing a valuable tool for machine learning engineers working with specialized or low-resource domains.

### Enhancing synthetic dataset diversity

Generating diverse datasets for training models can be challenging, particularly when dealing with specific applications such as legal document classification or children's story generation. One effective approach to address this challenge is by using prompt engineering to create varied synthetic datasets.

Consider the task of generating children's stories—a study by [Eldan et al. (2023)](https://arxiv.org/abs/2305.07759) provides a useful framework. Each story is composed of a few paragraphs and must cover a child's vocabulary and knowledge. The primary challenge in using LLMs for generating such training data is ensuring diversity in the dataset. Even with high generation temperatures, models can produce repetitive content, which limits the dataset's variability.

To address this, Eldan et al. developed a method using a predefined vocabulary of around 1,500 basic words, including nouns, verbs, and adjectives. By randomly selecting one verb, noun, and adjective for each story generation, the approach ensures a broader range of vocabulary and concept combinations. Additionally, they incorporated various story features like dialogue or plot twists, randomly choosing these features for each story.

**Example prompt for story generation**

To create diverse stories, you can use a prompt like this:

**Prompt:** Write a short story (3-5 paragraphs) using very simple words that a 3-year-old child would understand. The story should include the verb "{random.choice(verbs_list)}", the noun "{random.choice(nouns_list)}", and the adjective "{random.choice(adjectives_list)}". The story should have the following features: {random.choice(features_list)}, {random.choice(features_list)}. Remember to use only simple words!

**Filled-in Example:**  
**Prompt:** Write a short story (3-5 paragraphs) using very simple words that a 3-year-old child would understand. The story should include the verb "decorate", the noun "thunder", and the adjective "ancient". The story should contain at least one dialogue and have a bad ending. Remember to use only simple words!

**Output:** Once upon a time, in an ancient house, there lived a girl named Lily. She loved to decorate her room with pretty things. One day, she found a big box in the attic. She opened it and saw many shiny decorations. Lily was very happy and decided to use them in her room. As Lily was decorating her room, the sky outside became dark. There was a loud thunder sound, and Lily got scared. She ran to her mom and said, "Mommy, the thunder is so loud!" Her mom hugged her and said, "Don't worry, it will pass soon." But the thunder did not stop. It got louder and louder, and the ancient house started to shake. Suddenly, the roof fell down on the pretty decorations. Lily was sad because her room was not pretty anymore. The end.

By generating prompts with varied parameters, the dataset becomes more diverse. The key steps are:

  1. **Identify variable parameters:** Determine which elements in your dataset should vary, such as words, features, or labels.
  2. **Generate entities:** Collect or generate a set of these parameters.
  3. **Randomize prompts:** Create diverse prompts by randomly selecting from the generated entities.
  4. **Train the model:** Use the generated dataset to train your local model.



This approach can be further refined by incorporating iterative or hierarchical generation techniques. For instance, start with generating a story summary or key sentences and use them to guide the final content creation.

**Example of hierarchical generation:**

**Prompt:** Summary: Lily and Timmy build a sandcastle but it gets knocked over by the wind. They find beauty in the broken sandcastle and play happily with a butterfly. Features: Dialogue, Foreshadowing, Twist Sentence: One day, she went to the park and saw a beautiful butterfly. Words: disagree, network, beautiful Story:

**Output:** Once upon a time, there was a little girl named Lily. She loved to play with her toys and friends. One day, she went to the park and saw a beautiful butterfly. She tried to catch it, but it flew away. Lily's friend Timmy came to play with her. They decided to build a sandcastle, but they disagreed on how to build it. Lily wanted it tall, while Timmy wanted it wide. They argued but finally agreed to make it both tall and wide. As they built the sandcastle, they saw ants building a network of tunnels. A gust of wind knocked over their sandcastle. They were sad, but the butterfly landed on the ruins, showing them its beauty even in its broken state. They smiled and played happily.

This method facilitates the generation of a large number of varied examples, which is especially useful for training classifiers on specific features like dialogue or plot twists.

Finally, it's crucial to validate the effectiveness of synthetic data. [Gunasekar et al. (2023)](https://arxiv.org/abs/2306.11644) demonstrated that high-quality synthetic data can be highly beneficial, especially when training smaller models for specialized tasks. Their work emphasizes the importance of diversity and clarity in training data to improve model performance.

### Case study on prompt engineering for job classification

In a detailed case study, [Clavié et al. (2023)](https://arxiv.org/abs/2303.07142) investigated prompt engineering techniques for a medium-scale text classification task, specifically determining whether a job listing qualifies as an "entry-level" position suitable for recent graduates. Their research utilized GPT-3.5 (gpt-3.5-turbo) and compared its performance against several other models, including the robust DeBERTa-V3.

**Key findings:**

  1. **Model performance:** GPT-3.5 outperformed all other models tested, including DeBERTa-V3, in this classification task. It also showed significant improvements over earlier versions of GPT-3 across various performance metrics. However, it required additional effort in output parsing, as its adherence to a strict template was less reliable compared to other variants.
  2. **Prompt techniques:**


  *     * **Few-Shot vs. Zero-Shot:** Few-shot Chain-of-Thought (CoT) prompting underperformed compared to Zero-shot prompting in this context. The Zero-shot approach consistently produced better results.
    * **Impact of prompt design:** The design of the prompt had a substantial effect on performance. For instance, using a straightforward prompt to classify a job resulted in an F1 score of 65.6, while a well-engineered prompt elevated this score to 91.7.
    * **Template adherence:** Attempts to enforce strict adherence to a template generally led to decreased performance. However, this issue was less pronounced in early tests with GPT-4, suggesting improvements in template adherence in newer models.
    * **Minor adjustments:** Small changes in prompt design had a significant impact on results. For example, addressing the model by a (human) name and referring to it as such increased the F1 score by 0.6 points.



**Prompt modifications tested:**

  * **Baseline:** Provide a job posting and ask if it is suitable for a graduate.
  * **CoT (Chain-of-Thought)** : Include a few examples of correct classifications before posing the question.
  * **Zero-CoT:** Request the model to reason step-by-step before giving an answer.
  * **Rawinst:** Add role and task instructions directly to the user message.
  * **Sysinst** : Include role and task instructions as a system message.
  * **Bothinst:** Split role instructions as a system message and task instructions as a user message.
  * **Mock:** Simulate a discussion where the model acknowledges the task instructions.
  * **Reit:** Reinforce key instructions by repeating them.
  * **Strict:** Direct the model to follow a specific template strictly.
  * **Loose:** Request only the final answer following a given template.
  * **Right:** Ensure the model reaches the correct conclusion.
  * **Info:** Provide additional context to address common reasoning errors.
  * **Name:** Assign a name to the model and refer to it accordingly.
  * **Pos:** Offer positive feedback to the model before querying.



The study underscores the importance of prompt design in achieving high classification accuracy and demonstrates that nuanced adjustments can significantly enhance model performance.

### Leveraging encapsulated prompts in GPT-based systems

Encapsulated prompts in GPT models can be compared to defining functions in a programming language. This technique involves creating reusable, named prompts that perform specific tasks based on the input provided. It offers a structured approach to interacting with GPT, allowing users to automate and streamline complex processes efficiently.

Encapsulated prompts are essentially predefined templates or functions that GPT can execute. Each prompt is given a unique name and is designed to handle particular tasks or questions. This method transforms the interaction with GPT into a more systematic and organized workflow, making it easier to manage and repeat tasks.

**How it works?**

  1. **Defining a prompt function:** The first step is to define a function by setting up a prompt that clearly outlines its purpose. This includes specifying the function’s name, the type of input it requires, and the rules or instructions for processing that input. For instance, if you need a function to translate text, you would define it with instructions on how to handle translation.
  2. **Using the function:** Once defined, you can use this function by providing the input according to the function's design. The GPT model will then process the input based on the predefined rules and return the desired output. This approach allows you to interact with GPT in a way that is both structured and repeatable.
  3. **Combining functions:** You can also chain multiple functions together to perform a series of tasks. For example, you might first use a function to translate text and then apply another function to enhance or correct that text. This combination can automate more complex workflows, providing a seamless way to handle multiple steps in a process.



**Benefits of encapsulated prompts:**

  * **Consistency:** Provides a consistent method for performing specific tasks, reducing variability in output.
  * **Efficiency:** Streamlines repetitive tasks, saving time and effort.
  * **Organization:** Facilitates a more structured approach to interacting with GPT, making it easier to manage complex workflows.



**Tips for effective use:**

  * **Clear instructions:** Ensure that each function is clearly defined with specific input requirements and processing rules.
  * **Concise outputs:** Instruct GPT to provide only the necessary output to avoid excessive information.



By employing encapsulated prompts, users can significantly enhance their interaction with GPT, turning it into a powerful tool for automating and optimizing various tasks.

### Task decomposition

Task decomposition in prompt engineering involves breaking down complex problems into smaller, more manageable subtasks for an AI model to handle sequentially. Instead of asking the model to solve a complicated problem all at once, you guide it through a series of simpler steps. For example, if you wanted an AI to analyze a company's financial health, you might start with a prompt asking for key financial metrics, then follow up with prompts to calculate specific ratios, interpret those ratios, and finally summarize the overall financial condition.

This approach improves accuracy and allows the model to tackle more complex tasks than it could in a single prompt. It also provides more transparency in the AI's reasoning process. However, it requires careful planning to ensure that each subtask logically flows into the next and that the overall context is maintained throughout the decomposed steps.

## Better prompts, Better results: Tips for successful prompt engineering

To refine your approach in prompt engineering, consider these practical strategies:

**Analyze model responses:** Examine how the model responds to different prompts. This will help you adjust your prompts to get more accurate and relevant answers.

**Leverage user feedback:** Use feedback on how well the AI answers your prompts to make improvements. This can help you fine-tune the prompts for better performance.

**Adapt to model updates:** Keep up with changes in model architecture and new features. Adjust your prompts to make the most of these updates and enhancements.

**Collaborate and seek input:** Engage with others to gain new insights into prompt engineering. Participate in forums, social media groups, and professional networks to get advice and share experiences.

**Experiment with different prompt strategies:** Test various prompt techniques to find what works best in different situations. This flexibility can help you apply prompts more effectively across various contexts.

**Refine through iteration:** Regularly revise and test your prompts based on the outcomes they produce. Iteration helps in fine-tuning prompts to better meet your needs.

**Document your process:** Keep a record of the prompts you use and their effectiveness. Documenting your approach helps track what works well and facilitates future improvements.

**Understand model limitations:** Be aware of the model’s limitations and design your prompts accordingly. This understanding will help you set realistic expectations and get better results.

**Stay informed about best practices:** Keep up with best practices in prompt engineering by following industry updates and resources. Staying informed will help you apply the latest techniques and approaches.

**Build a knowledge base:** Create a repository of effective prompts and strategies that you can reference. This knowledge base can serve as a valuable resource for future prompt engineering tasks.

## Risks associated with prompting and solutions

As we harness the power of large language models and explore their capabilities, it is important to acknowledge the risks and potential misuses associated with prompting. While well-crafted prompts can yield impressive results, it is crucial to understand the potential pitfalls and safety considerations when using LLMs for real-world applications.

This section sheds light on the risks and misuses of LLMs, particularly through techniques like prompt injections. It also addresses harmful behaviors that may arise and provides insights into mitigating these risks through effective prompting techniques. Additionally, topics such as generalizability, calibration, biases, social biases, and factuality are explored to foster a comprehensive understanding of the challenges involved in working with LLMs.

By recognizing these risks and adopting responsible practices, we can navigate the evolving landscape of LLM applications while promoting ethical and safe use of these powerful language models.

### Adversarial prompting

Adversarial prompting refers to the intentional manipulation of prompts to exploit vulnerabilities or biases in language models, resulting in unintended or harmful outputs. Adversarial prompts aim to trick or deceive the model into generating misleading, biased, or inappropriate responses.

  * **Prompt injection:** Prompt injection is a technique used in adversarial prompting where additional instructions or content is inserted into the prompt to influence the model's behavior. By injecting specific keywords, phrases, or instructions, the model's output can be manipulated to produce desired or undesired outcomes. Prompt injection can be used to introduce biases, generate offensive or harmful content, or manipulate the model's understanding of the task.
  * **Prompt leaking:** Prompt leaking occurs when sensitive or confidential information unintentionally gets exposed in the model's response. This can happen when the model incorporates parts of the prompt, including personally identifiable information, into its generated output. Prompt leaking poses privacy and security risks, as it may disclose sensitive data to unintended recipients or expose vulnerabilities in the model's handling of input prompts.
  * **Jailbreaking:** In the context of prompt engineering, jailbreaking refers to bypassing or overriding safety mechanisms put in place to restrict or regulate the behavior of language models. It involves manipulating the prompt in a way that allows the model to generate outputs that may be inappropriate, unethical, or against the intended guidelines. Jailbreaking can lead to the generation of offensive content, misinformation, or other undesirable outcomes.



Overall, adversarial prompting techniques like prompt injection, prompt leaking, and jailbreaking highlight the importance of responsible and ethical prompt engineering practices. It is essential to be aware of the potential risks and vulnerabilities associated with language models and to take precautions to mitigate these risks while ensuring the safe and responsible use of these powerful AI systems.

#### Defense tactics for adversarial prompting

  * **Add defense in the instruction:** One defense tactic is to explicitly enforce the desired behavior through the instruction given to the model. While this approach is not foolproof, it emphasizes the power of well-crafted prompts in guiding the model towards the intended output.
  * **Parameterize prompt components:** Inspired by techniques used in SQL injection, one potential solution is to parameterize different components of the prompt, separating instructions from inputs and handling them differently. This approach can lead to cleaner and safer solutions, although it may come with some trade-offs in terms of flexibility.
  * **Quotes and additional formatting:** Escaping or quoting input strings can provide a workaround to prevent certain prompt injections. This tactic, suggested by Riley, helps maintain robustness across phrasing variations and highlights the importance of proper formatting and careful consideration of prompt structure.
  * **Adversarial prompt detector:** Language models themselves can be leveraged to detect and filter out adversarial prompts. By fine-tuning or training an LLM specifically for detecting such prompts, it is possible to incorporate an additional layer of defense to mitigate the impact of adversarial inputs.
  * **Selecting model types:** Choosing the appropriate model type can also contribute to defense against prompt injections. For certain tasks, using fine-tuned models or creating k-shot prompts for non-instruct models can be effective. Fine-tuning a model on a large number of examples can help improve robustness and accuracy, reducing reliance on instruction-based models.
  * **Guardrails and safety measures:** Some language models, like [ChatGPT](https://www.leewayhertz.com/build-an-app-with-chatgpt/), incorporate guardrails and safety measures to prevent malicious or dangerous prompts. While these measures provide a level of protection, they are not perfect and can still be susceptible to novel adversarial prompts. It is important to recognize the trade-off between safety constraints and desired behaviors.



### Factuality

It is worth noting that the field of prompt engineering and defense against adversarial prompting is an evolving area, and more research and development are needed to establish robust and comprehensive defense tactics against text-based attacks. Factuality is a significant risk in prompting as LLMs can generate responses that appear coherent and convincing but may lack accuracy. To address this, there are several solutions that can be employed:

  * **Provide ground truth:** Including reliable and factual information as part of the context can help guide the model to generate more accurate responses. This can involve referencing related articles, excerpts from reliable sources, or specific sections from Wikipedia entries. By incorporating verified information, the model is less likely to produce fabricated or inconsistent responses.
  * **Control response diversity:** Modifying the probability parameters of the model can influence the diversity of its responses. By decreasing the probability values, the model can be guided towards generating more focused and factually accurate answers. Additionally, explicitly instructing the model to acknowledge uncertainty by admitting when it doesn't possess the required knowledge can also mitigate the risk of generating false information.
  * **Provide examples in the prompt:** Including a combination of questions and responses in the prompt can guide the model to differentiate between topics it is familiar with and those it is not. By explicitly demonstrating examples of both known and unknown information, the model can better understand the boundaries of its knowledge and avoid generating false or speculative responses.



These solutions help address the risk of factuality in prompting by promoting more accurate and reliable output from LLMs. However, it is important to continuously evaluate and refine the prompt engineering strategies to ensure the best possible balance between generating coherent responses and maintaining factual accuracy.

### Biases

Biases in LLMs pose a significant risk as they can lead to the generation of problematic and biased content. These biases can adversely impact the performance of the model in downstream tasks and perpetuate harmful stereotypes or discriminatory behavior. To address this, it is essential to implement appropriate solutions:

  * **Effective prompting strategies:** Crafting well-designed prompts can help mitigate biases to some extent. By providing specific instructions and context that encourage fairness and inclusivity, the model can be guided to generate more unbiased responses. Additionally, incorporating diverse and representative examples in the prompt can help the model learn from a broader range of perspectives, reducing the likelihood of biased output.
  * **Moderation and filtering:** Implementing robust moderation and filtering mechanisms can help identify and mitigate biased content generated by LLMs. This involves developing systems that can detect and flag potentially biased or harmful outputs in real-time. Human reviewers or content moderation teams can then review and address any problematic content, ensuring that biased or discriminatory responses are not propagated.
  * **Diverse training data:** Training LLMs on diverse datasets that encompass a wide range of perspectives and experiences can help reduce biases. By exposing the model to a more comprehensive set of examples, it learns to generate responses that are more balanced and representative. Regularly updating and expanding the training data with diverse sources can further enhance the model's ability to generate unbiased content.
  * **Post-processing and debiasing techniques:** Applying post-processing techniques to the generated output can help identify and mitigate biases. These techniques involve analyzing the model's responses for potential biases and adjusting them to ensure fairness and inclusivity. Debiasing methods can be employed to retrain the model, explicitly addressing and reducing biases in its output.



It is important to note that addressing biases in LLMs is an ongoing challenge, and no single solution can completely eliminate biases. It requires a combination of thoughtful prompt engineering, robust moderation practices, diverse training data, and continuous improvement of the underlying models. Close collaboration between researchers, practitioners, and communities is crucial to develop effective strategies and ensure responsible and unbiased use of LLMs.

## Key ethical considerations in prompt engineering for AI models

While prompt engineering unlocks immense potential in AI, it simultaneously introduces significant ethical considerations that require careful navigation. From perpetuating societal biases to enabling the spread of misinformation, the very techniques that empower us also have the potential for significant harm if not wielded responsibly. This section delves into these ethical complexities, offering concrete examples and highlighting the need for ongoing dialogue and responsible AI development practices.

### Bias and discrimination:

**Prompt bias:** The way you phrase a prompt can inadvertently introduce biases. For example, a prompt like "Write a story about a successful CEO" might implicitly favor male characters, reflecting societal biases.

**Data bias:** The data used to train large language models often contains societal biases, which can be amplified through prompt engineering. This can result in outputs that perpetuate harmful stereotypes.

**Ethical implications:** Bias in outputs can lead to discrimination in various areas, such as hiring, loan applications, or even criminal justice.

### Misinformation and manipulation:

**Generating fake news:** Prompt engineering can be used to create realistic-sounding fake news articles or social media posts. This can spread misinformation and sow discord.

**Manipulating public opinion:** Malicious actors can use prompt engineering to generate persuasive content that influences public opinion or promotes specific agendas.

**Ethical concerns:** The potential for misuse in spreading misinformation and manipulating public discourse raises serious ethical concerns.

### Privacy and security:

**Prompt engineering in private conversations:** Prompt engineering could be used to generate outputs based on private conversations, raising concerns about data privacy.

**Generating personal information:** LLM prompt engineering could be used to generate plausible personal information, like names, addresses, or even medical records.

**Ethical implications:** The potential for misuse in violating privacy and security is a major concern.

### Accessibility and equity:

**Limited access to resources:** The ability to effectively use prompt engineering requires access to powerful language models and technical expertise. This creates a potential for inequality.

**Potential for misuse:** Those with the resources and skills to use prompt engineering effectively could exploit others who lack access or understanding.

**Ethical considerations:** Ensuring equitable access to the benefits of prompt engineering is crucial for a just and inclusive society.

### Addressing ethical considerations:

**Awareness and education:** It's essential to be aware of the potential for bias and misuse in prompt engineering.

**Responsible design:** Prioritize ethical considerations in prompt design, minimizing biases and fostering responsible use.

**Transparency and accountability:** Openly disclose the methods and limitations of prompt engineering to ensure transparency and accountability.

**Collaboration and regulation:** Collaboration between researchers, developers, and policymakers is necessary to establish guidelines and regulations for ethical prompt engineering.

In conclusion, prompt engineering offers immense potential but requires careful consideration of its ethical implications. By being mindful of bias, misinformation, privacy, accessibility, and other concerns, we can harness this powerful tool responsibly and ethically.

## The future of prompt engineering

**Widespread demand across industries -** The integration of AI into diverse sectors will lead to increased demand for skilled prompt engineers. Industries from healthcare to entertainment will require experts to optimize AI models through effective prompt engineering.

**Advancements in automated prompt generation -** The future will see significant progress in automated prompt generation, enhancing AI’s ability to autonomously create and refine prompts. While human prompt engineers will remain essential, this advancement will play a crucial role in AI’s evolution.

**Real-time communication and translation -** Future LLM prompt engineering advancements will facilitate real-time language translation and multilingual communication. By providing context across languages, AI will enable seamless communication, accounting for dialects and cultural nuances.

**Multimodal interfaces -** The development of multimodal interfaces—incorporating speech, eye tracking, touch, and gestures—will transform prompt engineering. These interfaces will create more intuitive and accessible interactions with AI systems.

**Edge computing -** Edge computing will enhance prompt engineering by enabling local data processing, which reduces latency and improves response times. For instance, devices like Google Pixel Buds use edge computing for real-time language translation without needing constant internet access.

**Personalization in prompt engineering -** Personalization will become a key focus in prompt engineering, with systems designed to adapt to individual user preferences and behaviors. Examples include recommendation systems that tailor suggestions based on user history.

**Adaptive prompting -** Adaptive prompting will involve dynamically adjusting prompts based on user feedback. AI models will learn from interactions to generate responses that better align with user needs and preferences.

**Data efficiency and sample efficiency -** Research will focus on improving data and sample efficiency in prompt engineering. Techniques such as few-shot learning and data augmentation will optimize performance while reducing reliance on extensive datasets.

## Endnote

The future of language model learning is deeply intertwined with the ongoing evolution of prompt engineering. As we stand on the threshold of this technological transformation, the vast and untapped potential of prompt engineering is coming into focus. It serves as a bridge between the complex world of AI and the intricacy of human language, facilitating communication that is not just effective, but also intuitive and human-like.

In the realm of LLM, well-engineered prompts play a pivotal role. They are the steering wheel guiding the direction of machine learning models, helping them navigate through the maze of human languages with precision and understanding. As AI technologies become more sophisticated and integrated into our daily lives - from voice assistants on our phones to AI chatbots in customer service - the role of prompt engineering in crafting nuanced, context-aware prompts have become more important than ever.

Moreover, as the field of LLM expands into newer territories like automated content creation, data analysis, and even healthcare diagnostics, prompt engineering will be at the helm, guiding the course. It's not just about crafting questions for AI to answer; it's about understanding the context, the intent, and the desired outcome, and encoding all of that into a concise, effective prompt.

Investing time, research, and resources into prompt engineering today will have a ripple effect on our AI-enabled future. It will fuel advancements in LLM and lay the groundwork for AI technologies we can't even envision yet. The future of LLM, and indeed, the future of our increasingly AI-integrated world, rests in the hands of skilled prompt engineers.

_Enhance your LLM 's power and performance with prompt engineering. To harness the power of prompt engineering, [hire LeewayHertz's LLM development services](https://www.leewayhertz.com/large-language-model-development-company/) today and ensure business success in today's AI-centric world!_

**Listen to the article**

What is Chainlink VRF

What is Chainlink VRF

What is Chainlink VRF


#### Author’s Bio

![Akash Takyar](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20180%20180'%3E%3C/svg%3E)![Akash Takyar](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/16213612/Akash-portrait-2.webp)

Akash Takyar [![LinkedIn](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![LinkedIn](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2018/04/12013804/LinkedIn.png)](https://www.linkedin.com/in/akashtakyar/)

CEO LeewayHertz

Akash Takyar is the founder and CEO of LeewayHertz. With a proven track record of conceptualizing and architecting 100+ user-centric and scalable solutions for startups and enterprises, he brings a deep understanding of both technical and user experience aspects.  
Akash's ability to build enterprise-grade technology solutions has garnered the trust of over 30 Fortune 500 companies, including Siemens, 3M, P&G, and Hershey's. Akash is an early adopter of new technology, a passionate technology enthusiast, and an investor in AI and IoT startups.

[Write to Akash](mailto:info@leewayhertz.com)

#### Related Services

[![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/09000305/Hire-Prompt-Engineers.png)](https://www.leewayhertz.com/hire-prompt-engineers/)

#### Hire Prompt Engineers

Transform your generative AI projects with our expert prompt engineering. Achieve unparalleled results with OpenAI, Midjourney, and other generative AI models.

[Explore Service](https://www.leewayhertz.com/hire-prompt-engineers/)

## Start a conversation by filling the form

Once you let us know your requirement, our technical expert will schedule a call and discuss your idea in detail post sign of an NDA.  
**All information will be kept confidential.**

__This field is hidden when viewing the form

OID

 __This field is hidden when viewing the form

Campaign ID

 __This field is hidden when viewing the form

Source

 __This field is hidden when viewing the form

Lead Source Description

First Name(Required)

Last Name(Required)

Company Email(Required)

Company Name(Required)

Job Title(Required)

Country(Required)

Select a country*United StatesUnited KingdomAfghanistanAlbaniaAlgeriaAmerican SamoaAndorraAngolaAntigua and BarbudaArgentinaArmeniaAustraliaAustriaAzerbaijanBahamasBahrainBangladeshBarbadosBelarusBelgiumBelizeBeninBermudaBhutanBoliviaBosnia and HerzegovinaBotswanaBrazilBruneiBulgariaBurkina FasoBurundiCambodiaCameroonCanadaCape VerdeCayman IslandsCentral African RepublicChadChileChinaColombiaComorosCongo, Democratic Republic of theCongo, Republic of theCosta RicaCôte d'IvoireCroatiaCubaCuraçaoCyprusCzech RepublicDenmarkDjiboutiDominicaDominican RepublicEast TimorEcuadorEgyptEl SalvadorEquatorial GuineaEritreaEstoniaEthiopiaFaroe IslandsFijiFinlandFranceFrench PolynesiaGabonGambiaGeorgiaGermanyGhanaGreeceGreenlandGrenadaGuamGuatemalaGuineaGuinea-BissauGuyanaHaitiHondurasHong KongHungaryIcelandIndiaIndonesiaIranIraqIrelandIsraelItalyJamaicaJapanJordanKazakhstanKenyaKiribatiNorth KoreaSouth KoreaKosovoKuwaitKyrgyzstanLaosLatviaLebanonLesothoLiberiaLibyaLiechtensteinLithuaniaLuxembourgMacedoniaMadagascarMalawiMalaysiaMaldivesMaliMaltaMarshall IslandsMauritaniaMauritiusMexicoMicronesiaMoldovaMonacoMongoliaMontenegroMoroccoMozambiqueMyanmarNamibiaNauruNepalNetherlandsNew ZealandNicaraguaNigerNigeriaNorthern Mariana IslandsNorwayOmanPakistanPalauPalestine, State ofPanamaPapua New GuineaParaguayPeruPhilippinesPolandPortugalPuerto RicoQatarRomaniaRussiaRwandaSaint Kitts and NevisSaint LuciaSaint Vincent and the GrenadinesSamoaSan MarinoSao Tome and PrincipeSaudi ArabiaSenegalSerbiaSeychellesSierra LeoneSingaporeSint MaartenSlovakiaSloveniaSolomon IslandsSomaliaSouth AfricaSpainSri LankaSudanSudan, SouthSurinameSwazilandSwedenSwitzerlandSyriaTaiwanTajikistanTanzaniaThailandTogoTongaTrinidad and TobagoTunisiaTurkeyTurkmenistanTuvaluUgandaUkraineUnited Arab EmiratesUruguayUzbekistanVanuatuVatican CityVenezuelaVietnamVirgin Islands, BritishVirgin Islands, U.S.YemenZambiaZimbabwe

Select a state/province(Required)

Select a state/province*AlabamaAlaskaArizonaArkansasCaliforniaColoradoConnecticutDelawareDistrict of ColumbiaFloridaGeorgiaHawaiiIdahoIllinoisIndianaIowaKansasKentuckyLouisianaMaineMarylandMassachusettsMichiganMinnesotaMississippiMissouriMontanaNebraskaNevadaNew HampshireNew JerseyNew MexicoNew YorkNorth CarolinaNorth DakotaOhioOklahomaOregonPennsylvaniaRhode IslandSouth CarolinaSouth DakotaTennesseeTexasUtahVermontVirginiaWashingtonWest VirginiaWisconsinWyomingArmed Forces AmericasArmed Forces EuropeArmed Forces Pacific

Select a state/province(Required)

Select a state/province*AvonBedfordshireBerkshireBordersBristolBuckinghamshireCambridgeshireChannel IslandsCheshireClevelandCornwallCumbriaDerbyshireDevonDorsetDurhamEast Riding of YorkshireEast SussexEssexGloucestershireGreater ManchesterHampshireHerefordshireHertfordshireHumbersideIsle of ManIsle of WightIsles of ScillyKentLancashireLeicestershireLincolnshireLondonMerseysideMiddlesexNorfolkNorth YorkshireNorthamptonshireNorthumberlandNottinghamshireOxfordshireRutlandShropshireSomersetSouth YorkshireStaffordshireSuffolkSurreyTyne and WearWarwickshireWest MidlandsWest SussexWest YorkshireWiltshireWorcestershire

Select a state/province(Required)

Select a state/province*Australian Capital TerritoryNorthern TerritoryNew South WalesQueenslandSouth AustraliaTasmaniaVictoriaWestern Australia

Select a state/province(Required)

Select a state/province*AlbertaBritish ColumbiaManitobaNew BrunswickNewfoundland and LabradorNova ScotiaNorthwest TerritoriesOntarioPrince Edward IslandQuebecSaskatchewanYukon Territory

Select a state/province(Required)

Select a state/province*County AntrimCounty ArmaghCounty DownCounty FermanaghCounty LondonderryCounty TyroneCounty CavanCounty DonegalCounty MonaghanCounty DublinCounty CarlowCounty KildareCounty KilkennyCounty LaoisCounty LongfordCounty LouthCounty MeathCounty OffalyCounty WestmeathCounty WexfordCounty WicklowCounty GalwayCounty MayoCounty LeitrimCounty RoscommonCounty SligoCounty ClareCounty CorkCounty KerryCounty LimerickCounty TipperaryCounty Waterford

Select a state/province(Required)

Select a state/province*DrenteFlevolandFrieslandGelderlandGroningenLimburgNoord BrabantNoord HollandOverijsselUtrechtZuid HollandZeeland

Comments(Required)

Comments

This field is for validation purposes and should be left unchanged.

Δ

## Related Insights

[![Comparison of Large Language Models \(LLMs\): A detailed analysis](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201080%20675'%3E%3C/svg%3E)![Comparison of Large Language Models \(LLMs\): A detailed analysis](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/04/22234632/Comparison-of-LLMs-02.png)](https://www.leewayhertz.com/comparison-of-llms/)

##  [Comparison of Large Language Models (LLMs): A detailed analysis](https://www.leewayhertz.com/comparison-of-llms/)

Large Language Models (LLMs) have emerged as a cornerstone in the advancement of artificial intelligence, transforming our interaction with technology and our ability to process and generate human language.

[read more](https://www.leewayhertz.com/comparison-of-llms/)

[![Actionable AI: An evolution from Large Language Models to Large Action Models](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201080%20675'%3E%3C/svg%3E)![Actionable AI: An evolution from Large Language Models to Large Action Models](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/03/02050435/Large-Action-Models-Feature.png)](https://www.leewayhertz.com/actionable-ai-large-action-models/)

##  [Actionable AI: An evolution from Large Language Models to Large Action Models](https://www.leewayhertz.com/actionable-ai-large-action-models/)

Actionable AI not only analyzes data but also uses those insights to drive specific, automated actions. 

[read more](https://www.leewayhertz.com/actionable-ai-large-action-models/)

[![Federated learning: Unlocking the potential of secure, distributed AI](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201080%20675'%3E%3C/svg%3E)![Federated learning: Unlocking the potential of secure, distributed AI](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/01/17031621/federated-machine-learning.png)](https://www.leewayhertz.com/federated-learning/)

##  [Federated learning: Unlocking the potential of secure, distributed AI](https://www.leewayhertz.com/federated-learning/)

Federated learning aims to train a unified model using data from multiple sources without the need to exchange the data itself.

[read more](https://www.leewayhertz.com/federated-learning/)

[![Optimize to actualize: The impact of hyperparameter tuning on AI](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201080%20675'%3E%3C/svg%3E)![Optimize to actualize: The impact of hyperparameter tuning on AI](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/01/11000044/Hyperparameter-tuning.png)](https://www.leewayhertz.com/hyperparameter-tuning/)

##  [Optimize to actualize: The impact of hyperparameter tuning on AI](https://www.leewayhertz.com/hyperparameter-tuning/)

Hyperparameter tuning is a critical aspect of machine learning, involving configuration variables that significantly influence the training process of a model.

[read more](https://www.leewayhertz.com/hyperparameter-tuning/)

[![Journey to AGI: Exploring the next frontier in artificial intelligence](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201080%20675'%3E%3C/svg%3E)![Journey to AGI: Exploring the next frontier in artificial intelligence](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/01/08015909/AGI.png)](https://www.leewayhertz.com/artificial-general-intelligence/)

##  [Journey to AGI: Exploring the next frontier in artificial intelligence](https://www.leewayhertz.com/artificial-general-intelligence/)

Artificial General Intelligence represents a significant leap in the evolution of artificial intelligence, characterized by capabilities that closely mirror the intricacies of human intelligence.

[read more](https://www.leewayhertz.com/artificial-general-intelligence/)

[![How attention mechanism’s selective focus fuels breakthroughs in AI](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201080%20675'%3E%3C/svg%3E)![How attention mechanism’s selective focus fuels breakthroughs in AI](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/08031657/Attention-mechanism-27190-02.svg)](https://www.leewayhertz.com/attention-mechanism/)

##  [How attention mechanism's selective focus fuels breakthroughs in AI](https://www.leewayhertz.com/attention-mechanism/)

The attention mechanism significantly enhances the model's capability to understand, process, and predict from sequence data, especially when dealing with long, complex sequences.

[read more](https://www.leewayhertz.com/attention-mechanism/)

[![What is LLMOps? Exploring the fundamentals and significance of large language model operations](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201080%20675'%3E%3C/svg%3E)![What is LLMOps? Exploring the fundamentals and significance of large language model operations](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/08031721/LLMOps-applications.svg)](https://www.leewayhertz.com/what-is-llmops/)

##  [What is LLMOps? Exploring the fundamentals and significance of large language model operations](https://www.leewayhertz.com/what-is-llmops/)

LLMOps, or Large Language Model Operations, encompass the practices, techniques, and tools used to deploy, monitor, and maintain LLMs effectively.

[read more](https://www.leewayhertz.com/what-is-llmops/)

[![Ensuring ML model accuracy and adaptability through model validation techniques](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201080%20675'%3E%3C/svg%3E)![Ensuring ML model accuracy and adaptability through model validation techniques](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/07033508/model-validation-techniques.svg)](https://www.leewayhertz.com/model-validation-in-machine-learning/)

##  [Ensuring ML model accuracy and adaptability through model validation techniques](https://www.leewayhertz.com/model-validation-in-machine-learning/)

As businesses lean heavily on data-driven decisions, it's not an exaggeration to say that a company's success may very well hinge on the strength of its model validation techniques.

[read more](https://www.leewayhertz.com/model-validation-in-machine-learning/)

[![ModelOps: An overview, use cases and benefits](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201080%20675'%3E%3C/svg%3E)![ModelOps: An overview, use cases and benefits](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/06030322/What-is-ModelOps.svg)](https://www.leewayhertz.com/what-is-modelops/)

##  [ModelOps: An overview, use cases and benefits](https://www.leewayhertz.com/what-is-modelops/)

ModelOps, short for Model Operations, is a set of practices and processes focusing on operationalizing and managing AI and ML models throughout their lifecycle. 

[read more](https://www.leewayhertz.com/what-is-modelops/)

[Show All Insights](https://www.leewayhertz.com/insights/)

#### LEEWAYHERTZ

  * [About Us](https://www.leewayhertz.com/about-us/)
  * [Careers](https://leewayhertz.com/careers/)
  * [Case Studies](https://www.leewayhertz.com/case-studies/)
  * [Work](https://www.leewayhertz.com/app-development-work/)
  * [Community](https://www.leewayhertz.com/community)
  * [Privacy Policy](https://www.leewayhertz.com/privacy-policy/)



#### PORTFOLIO

  * [Rackspace](https://www.leewayhertz.com/llm-powered-chatbot/)
  * [URC](https://www.leewayhertz.com/ai-driven-chatbot-for-dealer-support/)
  * [Scrut Automation](https://www.leewayhertz.com/scrut-llm-powered-app/)
  * [NSG](https://www.leewayhertz.com/ai-driven-label-verification-system/)
  * [AdPerfect](https://www.leewayhertz.com/ai-powered-saas-platform-for-advertisement-generation/)
  * [ZBrain](https://zbrain.ai/)



#### SERVICES

  * [AI Development](https://www.leewayhertz.com/ai-development-services-company/?utm_source=footer)
  * [AI Consulting](https://www.leewayhertz.com/ai-consulting-services-company/?utm_source=footer)
  * [Web3](https://www.leewayhertz.com/web3-development-company/?utm_source=footer)
  * [Blockchain](https://www.leewayhertz.com/blockchain-development-company/?utm_source=footer)
  * [Software Development](https://www.leewayhertz.com/software-development-company/?utm_source=footer)
  * [Hire AI Developers](https://www.leewayhertz.com/hire-ai-engineers/?utm_source=footer)



#### Generative AI

  * [Generative AI Development](https://www.leewayhertz.com/generative-ai-development-company/?utm_source=footer)
  * [Generative AI Consulting](https://www.leewayhertz.com/generative-ai-consulting-company/?utm_source=footer)
  * [Generative AI Integration](https://www.leewayhertz.com/generative-ai-integration-service/?utm_source=footer)
  * [LLM Development](https://www.leewayhertz.com/large-language-model-development-company/?utm_source=footer)
  * [AI Agent Development](https://www.leewayhertz.com/ai-agent-development-company/?utm_source=footer)
  * [AI Chatbot Development](https://www.leewayhertz.com/ai-chatbot-development-company/?utm_source=footer)



#### INDUSTRIES

  * [Finance](https://www.leewayhertz.com/fintech-ai-consulting-and-development-company/)
  * [Insurance](https://www.leewayhertz.com/insurance-ai-consulting-and-development-company/)
  * [Manufacturing](https://www.leewayhertz.com/manufacturing-ai-consulting-and-development-company/)
  * [Logistics](https://www.leewayhertz.com/logistics-ai-consulting-and-development-company/)
  * [Retail](https://www.leewayhertz.com/retail-ai-consulting-and-development-company/)
  * [Healthcare](https://www.leewayhertz.com/healthcare-ai-consulting-and-development-company/)



#### INSIGHTS

  * [AI Use Cases](https://www.leewayhertz.com/ai-use-cases-and-applications/)
  * [Conversational AI](https://www.leewayhertz.com/conversational-ai/)
  * [Private LLM](https://www.leewayhertz.com/build-private-llm/)
  * [AI in Finance](https://www.leewayhertz.com/ai-use-cases-in-banking-and-finance/)
  * [AI Document Processing](https://www.leewayhertz.com/intelligent-document-processing-idp/)
  * [AI Chatbot](https://www.leewayhertz.com/ai-chatbots/)



#### CONTACT US

[Get In Touch](https://www.leewayhertz.com/contact-us/)  
[sales@leewayhertz.com](mailto:sales@leewayhertz.com)  
[jobs@leewayhertz.com](mailto:jobs@leewayhertz.com)

![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2024%20300'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2021/07/30092405/Apple.svg)

 

![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2024%2028'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2021/07/30101534/Android.svg)

 

![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2024%2028'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2021/07/30101536/Windows.svg)

 

![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2072%2028'%3E%3C/svg%3E)![](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2021/07/30101532/Amazon-Web-services.svg)

 

  * [![X](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![X](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/09204442/xfootericon.svg)  
](https://twitter.com/LeewayHertz/)
  * [![Linkdin](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Linkdin](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/09204712/linkdinfootericon.svg)  
](https://in.linkedin.com/company/leewayhertz-technologies)
  * [![FB](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![FB](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/09204825/fbfootericon.svg)  
](https://www.facebook.com/LeewayHertz/)
  * [![telegram](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![telegram](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/09205348/telefootericon.svg)  
](https://t.me/LeewayHertzOfficial)



©2025 LeewayHertz. All Rights Reserved.

  


This website uses cookies to enhance site navigation and improve functionality, analyze site usage, and assist in our marketing and advertising efforts. [Read More](https://www.leewayhertz.com/privacy-policy/)  


  


This website uses cookies to enhance site navigation and improve functionality, analyze site usage, and assist in our marketing and advertising efforts. Please click "I accept cookies" to let us know you're okay with our use of all cookies. For more information please see the cookies section of our Privacy Policy. [Read More](https://www.leewayhertz.com/privacy-policy/)  


I ACCEPT COOKIES

Privacy & Cookies Policy

Close

#### Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.

Necessary 

Necessary

Always Enabled

Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information. 

Non-necessary 

Non-necessary

Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website. 

SAVE & ACCEPT

× 

[![Agents CTA](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Agents CTA](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/11/06062348/LHT-CTA.png)](https://zbrain.ai/agents/#lhtinsightpopup)

![Agents CTA](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Agents CTA](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2024/11/06212529/LHT-CTA-icon.svg)

× 

Follow Us

  * [![Twitter](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Twitter](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/16005726/twitter.svg)](https://twitter.com/leewayhertz)
  *   * [![Linkedin](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Linkedin](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/16005725/linkedin.svg)](https://www.linkedin.com/company/leewayhertz-technologies/)
  *   * [![Google](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)![Google](https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/11/16005725/google.svg)](https://news.google.com/publications/CAAqBwgKMIbRpQsw2du9Aw?hl=en-IN&gl=IN&ceid=IN%3Aen)
  * 


url: https://neptune.ai/blog/zero-shot-and-few-shot-learning-with-llms
title: Zero-Shot and Few-Shot Learning with LLMs
scrape_date: 2025-03-17T12:18:27.481906
content_type: markdown
hash: a77b9f71d2d32bf0b429a762fe4acc9f81bf2755b5aaeb0a4af06fa04f81fe9d
content:
[ Tell 120+K peers about your AI research → Learn more 💡 ![](https://neptune.ai/wp-content/themes/neptune/img/image-ratio-holder.svg) ](/neurips-2024)

[ ![logo](https://neptune.ai/wp-content/themes/neptune/img/logo-neptune.svg) ](https://neptune.ai "neptune.ai")

  * Product![](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-with-margin.svg)

    * ![Menu icon](https://neptune.ai/wp-content/uploads/2023/09/show.svg)Overview
      * [Walkthrough [2 min]](https://neptune.ai/walkthrough)
      * [Play with public sandbox](https://scale.neptune.ai/o/neptune/org/LLM-training-example/runs/compare?viewId=9d0e032a-5a78-4a0e-81d1-98e0a7c81a8f&detailsTab=metadata&dash=charts&type=run&experimentOnly=true&compare=u0MsW4a1PJIUJ75nglpjHa9XUKFfAmcBRbLhNatCHX20)
      * [Deployment options](https://neptune.ai/product/deployment-options)
    * ![Menu icon](https://neptune.ai/wp-content/uploads/2023/08/compare-1.svg)Compare
      * [Neptune vs WandB](https://neptune.ai/vs/wandb)
      * [Neptune vs MLflow](https://neptune.ai/vs/mlflow)
      * [Neptune vs TensorBoard](https://neptune.ai/vs/tensorboard)
      * [Other comparisons](https://neptune.ai/vs)

[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/07/bg-5.jpg?fit=768%2C432&ssl=1)Live Neptune projectPlay with a public example project that showcases Neptune's upcoming product release. It offers enhanced scalability and exciting new features. ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](https://scale.neptune.ai/o/neptune/org/LLM-training-example/runs/compare?viewId=9d0e032a-5a78-4a0e-81d1-98e0a7c81a8f&detailsTab=metadata&dash=charts&type=run&experimentOnly=true&compare=u0MsW4a1PJIUJ75nglpjHa9XUKFfAmcBRbLhNatCHX20)

  * Solutions![](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-with-margin.svg)

    * ![Menu icon](https://neptune.ai/wp-content/uploads/2023/08/bulb.svg)By role
      * [AI Researcher](https://neptune.ai/product/ai-researcher)
      * [ML Team Lead](https://neptune.ai/product/ml-team-lead)
      * [ML Platform Engineer](https://neptune.ai/product/ml-platform)
      * [Academia & Kagglers](https://neptune.ai/research)
    * ![Menu icon](https://neptune.ai/wp-content/uploads/2023/08/star.svg)By use case
      * [Monitor training](https://neptune.ai/product/monitor-training)
      * [Compare experiments](https://neptune.ai/product/compare-experiments)
      * [Collaborate with a team](https://neptune.ai/product/team-collaboration)
      * [Reports](https://neptune.ai/product/reports)

[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2022/11/deepsense.ai-logo-e1667753808279.png?fit=75%2C75&ssl=1)Case studyHow deepsense.ai Tracked and Analyzed 120K+ Models Using Neptune ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](/customers/deepsense-ai)[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2025/01/Artera-color-signet.png?fit=129%2C97&ssl=1)Case studyHow Neptune Helps Artera Bring AI Solutions to Market Faster ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](/customers/artera)

[See all case studies ![chevron](https://neptune.ai/wp-content/themes/neptune/img/icon-button-arrow-right.svg)](/customers)

  * [Documentation](https://docs.neptune.ai/)
  * Resources![](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-with-margin.svg)

    * Menu Item
      * [Blog](https://neptune.ai/blog)
      * [Experiment Tracking Learning Hub](https://neptune.ai/experiment-tracking-learn-hub)
      * [LLMOps Learning Hub](https://neptune.ai/llmops-learning-hub)
      * [MLOps Learning Hub](https://neptune.ai/mlops-learn-hub)
      * [100 Second Research Playlist](https://www.youtube.com/watch?v=_sKZsx7Iprg&list=PLKePQLVx9tOcAGAKvmRuQ5Y1_k6wqG0cX&pp=iAQB)

[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/12/Building-The-Most-Scalable-Experiment-Tracker-For-Foundation-Models.png?fit=768%2C403&ssl=1)ArticleFrom Research to Production: Building The Most Scalable Experiment Tracker For Foundation ModelsAurimas Griciūnas discusses the journey and challenges behind building the most scalable experiment tracker for foundation model training. ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](/blog/building-the-most-scalable-experiment-tracker-for-foundation-models)[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/11/reasercher-3-levels-challenge.webp?fit=768%2C432&ssl=1)VideoBreaking Down AI Research Across 3 Levels of DifficultyWe challenged AI/ML researchers to explain their work across 3 levels of difficulty: for a young learner, university student, and a fellow researcher.  ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](https://www.youtube.com/watch?v=zSEv3KBGlJQ&list=PLKePQLVx9tOfqC8ho2g_tQVxRga-XbIWa&index=3&t=9s)

  * [Pricing](https://neptune.ai/pricing)
  * [Enterprise](https://neptune.ai/product/enterprise)
  * Company![](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-with-margin.svg)

    * Menu Item
      * [About us](https://neptune.ai/about-us)
      * [Customers](https://neptune.ai/customers)
      * [Careers](https://neptune.ai/jobs)
      * [Security](https://security.neptune.ai/)
      * [Contact us](https://neptune.ai/contact-us)




![search](https://neptune.ai/wp-content/themes/neptune/img/icon-search.svg)

What do you want to find? 

Search

![cancel](https://neptune.ai/wp-content/themes/neptune/img/image-ratio-holder.svg)

[ Log in  ](https://app.neptune.ai/login) [ Sign up  ](https://app.neptune.ai/register) [ Contact us  ](https://neptune.ai/contact-us)

[![Home](/wp-content/themes/neptune/img/icon-breadcrumbs-home.svg)](https://neptune.ai/) > [Blog](https://neptune.ai/blog) > [LLMOps](https://neptune.ai/blog/category/llmops)

  * Topics![](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-with-margin.svg)

    * Categories
      * [ML Model Development](https://neptune.ai/blog/category/machine-learning-model-development)
      * [MLOps](https://neptune.ai/blog/category/mlops)
      * [LLMOps](https://neptune.ai/blog/category/llmops)
      * [ML Tools](https://neptune.ai/blog/category/machine-learning-tools)
      * [Computer Vision](https://neptune.ai/blog/category/computer-vision)
    * Categories
      * [Natural Language Processing](https://neptune.ai/blog/category/natural-language-processing)
      * [Reinforcement Learning](https://neptune.ai/blog/category/reinforcement-learning)
      * [Tabular Data](https://neptune.ai/blog/category/tabular-data)
      * [Time Series](https://neptune.ai/blog/category/time-series-forecasting)




[ LLMOps Learning Hub ](https://neptune.ai/llmops-learning-hub)

Search in Blog... 

![search](https://neptune.ai/wp-content/themes/neptune/img/image-ratio-holder.svg)

![search](https://neptune.ai/wp-content/themes/neptune/img/icon-search.svg) ![cancel](https://neptune.ai/wp-content/themes/neptune/img/image-ratio-holder.svg)

Search in Blog... 

![search](https://neptune.ai/wp-content/themes/neptune/img/image-ratio-holder.svg)

  * Product![](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-with-margin.svg)

    * ![Menu icon](https://neptune.ai/wp-content/uploads/2023/09/show.svg)Overview
      * [Walkthrough [2 min]](https://neptune.ai/walkthrough)
      * [Play with public sandbox](https://scale.neptune.ai/o/neptune/org/LLM-training-example/runs/compare?viewId=9d0e032a-5a78-4a0e-81d1-98e0a7c81a8f&detailsTab=metadata&dash=charts&type=run&experimentOnly=true&compare=u0MsW4a1PJIUJ75nglpjHa9XUKFfAmcBRbLhNatCHX20)
      * [Deployment options](https://neptune.ai/product/deployment-options)
    * ![Menu icon](https://neptune.ai/wp-content/uploads/2023/08/compare-1.svg)Compare
      * [Neptune vs WandB](https://neptune.ai/vs/wandb)
      * [Neptune vs MLflow](https://neptune.ai/vs/mlflow)
      * [Neptune vs TensorBoard](https://neptune.ai/vs/tensorboard)
      * [Other comparisons](https://neptune.ai/vs)

[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/07/bg-5.jpg?fit=768%2C432&ssl=1)Live Neptune projectPlay with a public example project that showcases Neptune's upcoming product release. It offers enhanced scalability and exciting new features. ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](https://scale.neptune.ai/o/neptune/org/LLM-training-example/runs/compare?viewId=9d0e032a-5a78-4a0e-81d1-98e0a7c81a8f&detailsTab=metadata&dash=charts&type=run&experimentOnly=true&compare=u0MsW4a1PJIUJ75nglpjHa9XUKFfAmcBRbLhNatCHX20)

  * Solutions![](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-with-margin.svg)

    * ![Menu icon](https://neptune.ai/wp-content/uploads/2023/08/bulb.svg)By role
      * [AI Researcher](https://neptune.ai/product/ai-researcher)
      * [ML Team Lead](https://neptune.ai/product/ml-team-lead)
      * [ML Platform Engineer](https://neptune.ai/product/ml-platform)
      * [Academia & Kagglers](https://neptune.ai/research)
    * ![Menu icon](https://neptune.ai/wp-content/uploads/2023/08/star.svg)By use case
      * [Monitor training](https://neptune.ai/product/monitor-training)
      * [Compare experiments](https://neptune.ai/product/compare-experiments)
      * [Collaborate with a team](https://neptune.ai/product/team-collaboration)
      * [Reports](https://neptune.ai/product/reports)

[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2022/11/deepsense.ai-logo-e1667753808279.png?fit=75%2C75&ssl=1)Case studyHow deepsense.ai Tracked and Analyzed 120K+ Models Using Neptune ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](/customers/deepsense-ai)[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2025/01/Artera-color-signet.png?fit=129%2C97&ssl=1)Case studyHow Neptune Helps Artera Bring AI Solutions to Market Faster ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](/customers/artera)

[See all case studies ![chevron](https://neptune.ai/wp-content/themes/neptune/img/icon-button-arrow-right.svg)](/customers)

  * [Documentation](https://docs.neptune.ai/)
  * Resources![](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-with-margin.svg)

    * Menu Item
      * [Blog](https://neptune.ai/blog)
      * [Experiment Tracking Learning Hub](https://neptune.ai/experiment-tracking-learn-hub)
      * [LLMOps Learning Hub](https://neptune.ai/llmops-learning-hub)
      * [MLOps Learning Hub](https://neptune.ai/mlops-learn-hub)
      * [100 Second Research Playlist](https://www.youtube.com/watch?v=_sKZsx7Iprg&list=PLKePQLVx9tOcAGAKvmRuQ5Y1_k6wqG0cX&pp=iAQB)

[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/12/Building-The-Most-Scalable-Experiment-Tracker-For-Foundation-Models.png?fit=768%2C403&ssl=1)ArticleFrom Research to Production: Building The Most Scalable Experiment Tracker For Foundation ModelsAurimas Griciūnas discusses the journey and challenges behind building the most scalable experiment tracker for foundation model training. ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](/blog/building-the-most-scalable-experiment-tracker-for-foundation-models)[ ![Menu thumbnail](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/11/reasercher-3-levels-challenge.webp?fit=768%2C432&ssl=1)VideoBreaking Down AI Research Across 3 Levels of DifficultyWe challenged AI/ML researchers to explain their work across 3 levels of difficulty: for a young learner, university student, and a fellow researcher.  ![chevron](https://neptune.ai/wp-content/themes/neptune/img/nav-article-arrow-right.svg) ](https://www.youtube.com/watch?v=zSEv3KBGlJQ&list=PLKePQLVx9tOfqC8ho2g_tQVxRga-XbIWa&index=3&t=9s)

  * [Pricing](https://neptune.ai/pricing)
  * [Enterprise](https://neptune.ai/product/enterprise)
  * Company![](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-with-margin.svg)

    * Menu Item
      * [About us](https://neptune.ai/about-us)
      * [Customers](https://neptune.ai/customers)
      * [Careers](https://neptune.ai/jobs)
      * [Security](https://security.neptune.ai/)
      * [Contact us](https://neptune.ai/contact-us)




[Neptune Blog](/blog)

#  Zero-Shot and Few-Shot Learning with LLMs 

![Author image](https://i0.wp.com/neptune.ai/wp-content/uploads/2022/11/Michal_Oleszak.jpeg?fit=1599%2C1406&ssl=1)

[ Michał Oleszak  ](https://neptune.ai/blog/author/michal-oleszak)

![](https://neptune.ai/wp-content/themes/neptune/img/icon-meta-time.svg) 7 min 

![](https://neptune.ai/wp-content/themes/neptune/img/icon-meta-date.svg) 25th September, 2024 

[LLMOps](https://neptune.ai/blog/category/llmops)

![](data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 3 2'%3E%3C/svg%3E)

###  TL;DR 

![](https://neptune.ai/wp-content/themes/neptune/img/blocks/note/list-arrow.svg)

Chatbots based on LLMs can solve tasks they were not trained to solve either out-of-the-box (zero-shot prompting) or when prompted with a couple of input-output pairs demonstrating how to solve the task (few-shot prompting).

![](https://neptune.ai/wp-content/themes/neptune/img/blocks/note/list-arrow.svg)

Zero-shot prompting is well-suited for simple tasks, exploratory queries, or tasks that only require general knowledge. It doesn’t work well for complex tasks that require context or when a very specific output form is needed.

![](https://neptune.ai/wp-content/themes/neptune/img/blocks/note/list-arrow.svg)

Few-shot prompting is useful when we need the model to “learn” a new concept or when a precise output form is required. It’s also a natural choice with very limited data (too little to train on) that could help the model to solve a task.

![](https://neptune.ai/wp-content/themes/neptune/img/blocks/note/list-arrow.svg)

If complex multi-step reasoning is needed, neither zero-shot nor few-shot prompting can be expected to yield good performance. In these cases, fine-tuning of the LLM will likely be necessary.

Chatbots based on Large Language Models (LLMs), such as OpenAI’s ChatGPT, show an astonishing capability to perform tasks for which they have not been explicitly trained. In some cases, they can do it out of the box. In others, the user must specify a few labeled examples for the model to pick up the pattern.

Two popular techniques for helping a Large Language Model solve a new task are zero-shot and few-shot prompting. In this article, we’ll explore how they work, see some examples, and discuss when to use (and, more importantly, when not to use) zero-shot and few-shot prompting.

## The role of zero-shot and few-shot learning in LLMs

The goal of [zero-shot and few-shot learning](/blog/understanding-few-shot-learning-in-computer-vision) is to get a machine-learning model to perform a new task it was not trained for. It is only natural to start by asking: what _are_ the LLMs trained to do?

![Diagram comparing pre-training to fine-tuning. In pre-training, the model predicts the next word, e.g., the United States’ first president was George -> Washington. In fine-tuning, the model produces a few answers, and the one that is accurate and polite is chosen.](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/03/Zero-shot-and-few-shot-learning-with-LLMs-1-1.png?resize=1200%2C628&ssl=1)_LLMs used in chatbot applications typically undergo two training stages. In pre-training, they learn to predict the next word. During fine-tuning, they learn to give specific responses_. | Source: Author

Most LLMs used in chatbots today undergo two stages of training:  


  * In the pre-training stage, the model is fed a large corpus of text and learns to predict the next word based on the previous words.  

  * In the fine-tuning stage, the next word predictor is adapted to behave as a chatbot, that is, to answer users' queries in a conversational manner and produce responses that meet human expectations.



Let’s see if OpenAI’s ChatGPT (based on GPT4) can finish a popular English-language pangram (a sentence containing all the letters of the alphabet):

![Screenshot of the ChatGPT interface. You: "quick brown fox jumps over the", ChatGPT: "lazy dog".
](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/03/Zero-shot-and-few-shot-learning-with-llms-1.png?resize=962%2C516&ssl=1)

As expected, it finishes the famous sentence correctly, likely having seen it multiple times in the pre-training data. If you’ve ever used ChatGTP, you’ll also know that chatbots appear to have vast factual knowledge and generally try to be helpful and avoid vulgarism.

  
But ChatGPT and similar LLM-backed chatbots can do so much more than that. They can solve many tasks they have never been trained to solve, such as translating between languages, detecting the sentiment in a text, or writing code.  


Getting chatbots to solve new tasks requires zero-shot and few-shot prompting techniques.

### Zero-shot prompting

Zero-shot prompting refers to simply asking the model to do something it was not trained to do. 

The word “zero” refers to giving the model no examples of how this new task should be solved. We just ask it to do it, and the Large Language Model will use the general understanding of the language and the information it learned during the training to generate the answer.

For example, suppose you ask a model to translate a sentence from one language to another. In that case, it will likely produce a decent translation, even though it was never explicitly trained for translation. Similarly, most LLMs can tell a negative-sounding sentence from a positively-sounding one without explicitly being trained in sentiment analysis.

### Few-shot prompting

Similarly, few-shot prompting means asking a Large Language Model to solve a new task while providing examples of how the task should be solved.

It is like passing a small sample of training data to the model through the query, allowing the model to learn from the user-provided examples. However, unlike during the pre-training or fine-tuning stages, the learning process does not involve updating the model’s weights. Instead, the model stays frozen but uses the provided context when generating its response. This context will typically be retained throughout a conversation, but the model cannot access the newly acquired information later.

Sometimes, specific variants of few-shot learning are distinguished, especially when evaluating and comparing model performance. “One-shot” means we provide the model with just one example, “two-shot” means we provide two examples – you get the gist.

![Examples of zero-shot and few-shot prompting. Zero-shot question: What does "LLM" stand for? Answer: {correct answer}. } Few-shot: cow-moo, cat-meow, dog-woof, duck-. Model: quack.](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/03/Zero-shot-and-few-shot-learning-with-LLMs-2.png?resize=1200%2C628&ssl=1)_In zero-shot prompting, the model answers based on its general knowledge. In few-shot prompting, it answers conditioning on examples provided in the prompt._ | Source: Author

### Is few-shot prompting the same as few-shot learning?

“Few-shot learning” and “zero-shot learning” are well-known concepts in machine learning that were studied long before LLMs appeared on the scene. In the context of LLMs, these terms are sometimes used interchangeably with “few-shot prompting” and “zero-shot prompting.” However, they are not the same.

Few-shot prompting refers to constructing a prompt consisting of a couple of examples of input-output pairs with the goal of providing an LLM with a pattern to pick up.

Few-shot learning is a model adaptation resulting from few-shot prompting, in which the model changes from being unable to solve the task to being able to solve it thanks to the provided examples.  
  
In the context of LLMs, the “learning” is temporary and only applies to a particular chat conversation. The model’s parameters are not updated, so it doesn’t retain the knowledge or capabilities.

[ ![](https://neptune.ai/wp-content/themes/neptune/img/icon-related--article.svg) Recommended  Understanding Few-Shot Learning in Computer Vision: What You Need to Know  Read also  ![](https://neptune.ai/wp-content/themes/neptune/img/icon-button-arrow-right.svg) ](/blog/understanding-few-shot-learning-in-computer-vision)

## Applications of zero-shot prompting LLMs

In zero-shot prompting, we rely on the model’s existing knowledge to generate responses. 

Consequently, zero-shot prompting makes sense for generic requests rather than for ones requiring highly specialized or proprietary knowledge.

### When to use zero-shot prompting

You can safely use zero-shot prompting in the following use cases:  


  * **Simple tasks** : If the task is simple, knowledge-based, and clearly defined, such as defining a word, explaining a concept, or answering a general knowledge question.  

  * **Tasks requiring general knowledge** : For tasks that rely on the model's pre-existing knowledge base, such as summarizing known information on a topic. They are more about clarifying, summarizing, or providing details on known subjects rather than exploring new areas or generating ideas. For example, “Who was the first person to climb Mount Everest?” or “Explain the process of photosynthesis.”  

  * **Exploratory queries** : When exploring a topic and wanting a broad overview or a starting point for research. These queries are less about seeking specific answers and more about getting a wide-ranging overview that can guide further inquiry or research. For example, “How do different cultures celebrate the new year?” or “What are the main theories in cognitive psychology?”  

  * **Direct instructions** : When you can provide clear, direct instruction that doesn't require examples for the model to understand the task. 



### When not to use zero-shot prompting

In the following situations, do not use zero-shot prompting:  


  * **Complex tasks requiring context** : If the task requires understanding nuanced context or specialized knowledge that the model is unlikely to have acquired during training.  

  * **Highly specific outcomes desired** : When you need a response tailored to a specific format, style, or set of constraints, the model may not be able to adhere to without guidance from input-output examples.



### Examples of zero-shot prompting use cases

Zero-shot prompting will get the job done for you in many simple NLP tasks, such as language translation or sentiment analysis.

As you can see in the screenshot below, translating a sentence from Polish to English is a piece of cake for ChatGPT:

![Screenshot of the ChatGPT interface. Chat is easily translating a sentence from Polish to English.](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/03/Zero-shot-and-few-shot-learning-with-llms-2.png?resize=1166%2C450&ssl=1)

Let’s try a zero-shot prompting-based strategy for sentiment analysis:

![Screenshot of the ChatGPT interface. Usage of a zero-shot prompting-based strategy for sentiment analysis.](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/03/Zero-shot-and-few-shot-learning-with-llms-3.png?resize=1292%2C788&ssl=1)

Again, the model got it right. With no explicit training for the task, ChatGPT was able to extract the sentiment from the text while avoiding pitfalls such as the first expression containing the word “good” even though the overall sentiment is negative. In the last example, which is somewhat more nuanced, the model even provided its reasoning behind the classification.

### Where zero-shot prompting fails

Let’s turn to two use cases where zero-shot prompting is insufficient. Recall that these are complex tasks requiring context and situations requiring a highly specific outcome.

Consider the following two prompts:  


  * “Explain the implications of the latest changes in quantum computing for encryption, considering current technologies and future prospects.”  

  * “Write a legal brief arguing the case for a specific, but hypothetical, scenario where an AI created a piece of art, and now there's a copyright dispute between the AI's developer and a gallery claiming ownership.”



To the adventurous readers over there, feel free to try these out with your LLM of choice! However, you’re rather unlikely to get anything useful as a result.

Here is why:

The first prompt about quantum computing demands an understanding of current, possibly cutting-edge developments in quantum computing and encryption technologies. Without specific examples or context, the LLM might not accurately reflect the latest research, advancements, or the nuanced implications for future technologies.

The second prompt, asking for a legal brief, requires the LLM to adhere to legal brief formatting and conventions, understand the legal intricacies of copyright law as it applies to AI (many of which are still subject to debate), and construct arguments based on hypothetical yet particular circumstances. A zero-shot prompt doesn't provide the model with the necessary guidelines or examples to generate a response that accurately meets all these detailed requirements.

## Applications of few-shot prompting

With few-shot prompting, the LLM conditions its response on the examples we provide. Hence, it makes sense to try it when it seems like just a few examples should be enough to discover a pattern or when we need a specific output format or style. However, a high degree of task complexity and latency restrictions are typical blockers for using few-shot prompting.

### When to use few-shot prompting

You can try prompting the model with a couple of examples in the following situations:  


  * **Zero-shot prompting is insufficient** :**** The model does not know how to perform the task well without any examples, but there is a reason to hope that just a few examples will suffice.  

  * **Limited training data is available** : When a few examples are all we have, fine-tuning the model is not feasible, and few-shot prompting might be the only way to get the examples across.  

  * **Custom formats or styles** : If you want the output to follow a specific format, style, or structure, providing examples can guide the model more effectively than trying to convey the desired outcome through words.  

  * **Teaching the model new concepts** : If you're trying to get the model to understand an idea it is unfamiliar with, a few examples can serve as a quick primer. Remember that this new knowledge is only retained for the conversation at hand, though!  

  * **Improving accuracy** : When precision is crucial, and you want to ensure the model clearly understands the task.



### When not to use few-shot prompting

In the following situations, you might want to decide against few-shot prompting:  


  * **General knowledge tasks** : For straightforward tasks that don't require specific formats or nuanced understanding, few-shot prompting might be overkill and unnecessarily complicate the query (unless, as discussed, accuracy is crucial).  

  * **Speed or efficiency is a priority** : Few-shot prompting requires more input, which can be slower to compose and process.  

  * **Insufficient examples** : If the task is too complex to explain in a few examples or if the specific examples you have available might confuse the model by introducing too much variability.  

  * **Complex reasoning tasks** : If the task requires a couple of reasoning steps, even a set of examples might not be enough for the LLM to get the pattern we are looking for.



### Examples of few-shot prompting use cases

Let’s examine examples where few-shot prompting proves highly effective.

#### Adapting tasks to specific styles

Imagine you work for a company that sells _Product B_. Your main competitor is _Product A_. You’ve collected some reviews from the internet, both on your product and the competing one. You want to get an idea of which product users consider to be better. To do so, you want to prompt the LLM to classify the sentiment of reviews for both products.

One way to solve this task is to manually craft a handful of examples such that:  


  * Good reviews of your product (B) are labeled as positive.
  * Bad reviews of your product (B) are labeled as negative.
  * Good reviews of the competing product (A) are labeled as positive.
  * Bad reviews of the competing product (A) are labeled as positive.



This should hopefully be enough for the model to see what you’re doing there.

![Screenshot of the ChatGPT interface. Usage of a few-shot prompting to steer the model into solving a conventional task \(sentiment classification\) in an unconventional way based on a specific label format.](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/03/Zero-shot-and-few-shot-learning-with-llms-4.png?resize=1378%2C1356&ssl=1)

Indeed, the model picked up the pattern correctly and predicted the good review of a competitor’s product as negative for us, and was even able to explain it:

> (…) positive sentiment expressions for Product A are labeled as "negative" and negative sentiment expressions are labeled as "positive" (and the conventional labeling for Product B). 

This was an example of how few-shot prompting allows us to steer the model into solving a conventional task (sentiment classification) in an unconventional way based on a specific label format.

#### Teaching an LLM new concepts

Few-shot prompting is particularly well-suited for teaching an LLM new or imaginary concepts. This can be useful when you need the model to discover patterns in your data that require understanding the quirks and details where general knowledge is useless.

Let’s see how we can use few-shot prompting to teach an LLM the basic grammar of a new language I have just invented, Blablarian. (It’s widely spoken in the Kingdom of Blabland if you’re curious.)

![Screenshot of the ChatGPT interface. Usege of a few-shot prompting to teach an LLM the basic grammar of a new \(imaginary\) language. ](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/03/Zero-shot-and-few-shot-learning-with-llms-5.png?resize=1346%2C1412&ssl=1)

As you can see, the model produced what must be regarded as a correct translation. It deciphered the meaning of the words and learned to distinguish between different pronouns. We can be sure this is purely in-context few-shot learning since there is no way Blablarian manuscripts could have made it into the model’s pre-training datasets.

This example illustrates the essence of few-shot learning well. Had we asked the model to translate the sentence “How old is he?” from English to Blablarian without providing any examples (that is, using zero-shot prompting), it wouldn’t have been able to do so simply because there is no such language as Blablarian. However, the model does have a general understanding of language and how grammar works. This knowledge is enough to pick up the patterns of a fake language I invented on the spot.

### Where few-shot prompting fails

Finally, let’s look at a situation where few-shot prompting won’t get us far.  
  
I will borrow this famous example that [has been circling around the internet](https://www.promptingguide.ai/techniques/cot) recently:

**Prompt** :

The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.A: The answer is False.The odd numbers in this group add up to an even number: 17, 10, 19, 4, 8, 12, 24.A: The answer is True.The odd numbers in this group add up to an even number: 16, 11, 14, 4, 8, 13, 24.A: The answer is True.The odd numbers in this group add up to an even number: 17, 9, 10, 12, 13, 4, 2.A: The answer is False.The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 7, 1. A:

 

**Response:**

The answer is True.

This answer is incorrect. A couple of examples are not enough to learn the pattern—the problem requires understanding several fundamental concepts and step-by-step reasoning. Even a significantly larger number of examples is unlikely to help.  
  
Arguably, this type of problem might not be solvable by pattern finding, and no prompt engineering can help.

But guess what: the LLMs of today can recognize that they face a type of problem they won’t be able to solve. These chatbots will then employ tools better suited for the particular task, just like if I asked you to multiply two large numbers and you would resort to a calculator.

OpenAI’s ChatGPT, for instance, instead of hallucinating a response, will produce a snippet of Python code that should answer the question. (This code is visible when you click on “Finished analyzing.”) ChatGPT will execute the generated code in an interpreter and provide the answer based on the code’s outputs. In this case, this approach led to a correct answer:

![Screenshot of the ChatGPT interface. Chat GPT producing a snippet of Python code that should answer the question. \(The code is visible after clicking “Finished analyzing.”\)](https://i0.wp.com/neptune.ai/wp-content/uploads/2024/03/Zero-shot-and-few-shot-learning-with-llms-6.png?resize=1406%2C970&ssl=1)

This “magic” is the consequence of OpenAI doing some work behind the scenes: they feed additional prompts to the LLM to ensure it knows when you use external tools such as the Python interpreter.

Note, however, that this is not “few-shot learning” anymore. The model did not use the examples provided. Indeed, it would have provided the same answer even in the zero-shot prompting setting.

[ ![](https://neptune.ai/wp-content/themes/neptune/img/icon-related--article.svg) See also  Strategies For Effective Prompt Engineering  Read more  ![](https://neptune.ai/wp-content/themes/neptune/img/icon-button-arrow-right.svg) ](/blog/prompt-engineering-strategies)

## Conclusion

This article delved into zero-shot and few-shot prompting with Large Language Models, highlighting capabilities, use cases, and limitations.

Zero-shot learning enables LLMs to tackle tasks they weren't explicitly trained for, relying solely on their pre-existing knowledge and general language understanding. This approach is ideal for simple tasks and exploratory queries, and when clear, direct instructions can be provided.

Few-shot learning allows LLMs to adapt to specific tasks, formats, or styles and improve accuracy for more complex queries by incorporating a small number of examples into the prompt.

However, both techniques have their limitations. Zero-shot prompting may not suffice for complex tasks requiring nuanced understanding or highly specific outcomes. Few-shot learning, while powerful, is not always the best choice for general knowledge tasks or when efficiency is a priority, and it may struggle with tasks too complex for a few examples to clarify.

As users and developers, understanding when and how to apply zero-shot and few-shot prompting can enable us to leverage the full potential of Large Language Models while navigating their limitations.

##  Was the article useful? 

![yes](https://neptune.ai/wp-content/themes/neptune/img/icon-article-rating--yes.svg) Yes  ![no](https://neptune.ai/wp-content/themes/neptune/img/icon-article-rating--no.svg) No 

![](https://neptune.ai/wp-content/themes/neptune/img/icon-bulb.svg) Suggest changes 




Your email Your message (optional)

This site is protected by reCAPTCHA and the Google [Privacy Policy](https://policies.google.com/privacy) and [Terms of Service](https://policies.google.com/terms) apply. 

I am familiar with the [Privacy Policy](https://neptune.staginglab.eu/privacy-policy)*

Submit

Δ

![](https://neptune.ai/wp-content/themes/neptune/img/blocks/i-box/header-icon.svg)

### **More about** Zero-Shot and Few-Shot Learning with LLMs 

####  Check out our  **product resources** and  **related articles** below: 

[ ![](https://neptune.ai/wp-content/themes/neptune/img/ibox-related.svg) Related article  LLMOps: What It Is, Why It Matters, and How to Implement It  Read more  ![chevron](https://neptune.ai/wp-content/themes/neptune/img/icon-button-arrow-right.svg) ](https://neptune.ai/blog/llmops) [ ![](https://neptune.ai/wp-content/themes/neptune/img/ibox-related.svg) Related article  LLM Fine-Tuning and Model Selection Using Neptune and Transformers  Read more  ![chevron](https://neptune.ai/wp-content/themes/neptune/img/icon-button-arrow-right.svg) ](https://neptune.ai/blog/llm-fine-tuning-and-model-selection-with-neptune-transformers) [ ![](https://neptune.ai/wp-content/themes/neptune/img/ibox-related.svg) Related article  Building LLM Applications With Vector Databases  Read more  ![chevron](https://neptune.ai/wp-content/themes/neptune/img/icon-button-arrow-right.svg) ](https://neptune.ai/blog/building-llm-applications-with-vector-databases) [ ![](https://neptune.ai/wp-content/themes/neptune/img/ibox-related.svg) Related article  LLM Evaluation For Text Summarization  Read more  ![chevron](https://neptune.ai/wp-content/themes/neptune/img/icon-button-arrow-right.svg) ](https://neptune.ai/blog/llm-evaluation-text-summarization)

###  Explore more content topics: 

[ Computer Vision ](https://neptune.ai/blog/category/computer-vision) [ General ](https://neptune.ai/blog/category/general) [ LLMOps ](https://neptune.ai/blog/category/llmops) [ ML Model Development ](https://neptune.ai/blog/category/machine-learning-model-development) [ ML Tools ](https://neptune.ai/blog/category/machine-learning-tools) [ MLOps ](https://neptune.ai/blog/category/mlops) [ Natural Language Processing ](https://neptune.ai/blog/category/natural-language-processing) [ Paper Reflections ](https://neptune.ai/blog/category/paper-reflections) [ Product Updates ](https://neptune.ai/blog/category/product-updates) [ Reinforcement Learning ](https://neptune.ai/blog/category/reinforcement-learning) [ Tabular Data ](https://neptune.ai/blog/category/tabular-data) [ Time Series ](https://neptune.ai/blog/category/time-series-forecasting)

About neptune.ai ![chevron](https://neptune.ai/wp-content/themes/neptune/img/image-ratio-holder.svg)

![](data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 1920 1080'%3E%3C/svg%3E) ![](https://neptune.ai/wp-content/themes/neptune/img/icon-play.svg)

Neptune is the experiment tracker purpose-built for foundation model training. 

It lets you monitor and visualize thousands of per-layer metrics—losses, gradients, and activations—at any scale. Drill down into logs and debug training issues fast. Keep your model training stable while reducing wasted GPU cycles. 

[ Play with a live project  ![](https://neptune.ai/wp-content/themes/neptune/img/image-ratio-holder.svg) ](https://demo.neptune.ai/) [ See Docs  ![](https://neptune.ai/wp-content/themes/neptune/img/image-ratio-holder.svg) ](https://docs.neptune.ai/)

Table of contents

![chevron](https://neptune.ai/wp-content/themes/neptune/img/icon-chevron-down-small.svg)

  1. The role of zero-shot and few-shot learning in LLMs
     * Zero-shot prompting
     * Few-shot prompting
  2. Applications of zero-shot prompting LLMs
  3. Applications of few-shot prompting
  4. Conclusion



![chevron](https://neptune.ai/wp-content/themes/neptune/img/popup-cancel.svg) ![chevron](https://neptune.ai/wp-content/themes/neptune/img/popup-more.svg)

Check also: [LLMOps: What It Is, Why It Matters, and How to Implement It](https://neptune.ai/blog/llmops)

##  **Monitor your model training at scale**

Join 60,000+ researchers and practitioners who use Neptune to debug training failures, spot anomalies, and compare experiments.

[ Try for free  ](https://app.neptune.ai/register) [ Get early access to Neptune Scale  ](/early-access)

[ Take an interactive product tour  ![](https://neptune.ai/wp-content/themes/neptune/img/image-ratio-holder.svg) ](https://scale.neptune.ai/o/neptune/org/LLM-training-example/runs/compare?viewId=9d0e032a-5a78-4a0e-81d1-98e0a7c81a8f&detailsTab=metadata&dash=charts&type=run&experimentOnly=true&compare=u0MsW4a1PJIUJ75nglpjHa9XUKFfAmcBRbLhNatCHX20)

![](https://neptune.ai/wp-content/themes/neptune/img/footer-post-illustration.svg)

#### Newsletter

Top articles, case studies, events (and more) in your inbox every month.

Get Newsletter

  * Product
    * [Walkthrough](https://neptune.ai/walkthrough)
    * [Documentation](https://docs.neptune.ai/)
    * [Deployment options](https://neptune.ai/product/deployment-options)
    * [Pricing](https://neptune.ai/pricing)
    * [Service status](https://status.neptune.ai/)


  * Solutions
    * [AI Researcher](https://neptune.ai/product/ai-researcher)
    * [ML Team Lead](https://neptune.ai/product/ml-team-lead)
    * [ML Platform Engineer](https://neptune.ai/product/ml-platform)
    * [Enterprise](https://neptune.ai/product/enterprise)
    * [Academic Research](https://neptune.ai/research)


  * Compare
    * [Neptune vs Weights & Biases](https://neptune.ai/vs/wandb)
    * [Neptune vs MLflow](https://neptune.ai/vs/mlflow)
    * [Neptune vs TensorBoard](https://neptune.ai/vs/tensorboard)
    * [Other comparisons](/vs)
    * [ML experiment tracking tools](/blog/best-ml-experiment-tracking-tools)


  * Community
    * [Blog](https://neptune.ai/blog)
    * [Experiment Tracking Learning Hub](https://neptune.ai/experiment-tracking-learn-hub)
    * [LLMOps Learning Hub](https://neptune.ai/llmops-learning-hub)
    * [MLOps Learning Hub](https://neptune.ai/mlops-learn-hub)
    * [How to Build an Experiment Tracker](https://neptune.ai/blog/build-experiment-tracking-tool)


  * Company
    * [Customers](https://neptune.ai/customers)
    * [About us](https://neptune.ai/about-us)
    * [Careers](https://neptune.ai/jobs)
    * [Security portal and SOC 2](https://security.neptune.ai/)
    * [Contact us](https://neptune.ai/contact-us)



[ ![social icon](https://neptune.ai/wp-content/uploads/2022/08/icon-linked-in.svg) ](https://www.linkedin.com/company/neptuneai) [ ![social icon](https://neptune.ai/wp-content/uploads/2022/08/icon-twitter.svg) ](https://twitter.com/neptune_ai) [ ![social icon](https://neptune.ai/wp-content/uploads/2022/08/icon-github.svg) ](https://github.com/neptune-ai) [ ![social icon](https://neptune.ai/wp-content/uploads/2022/08/icon-facebook.svg) ](https://www.facebook.com/neptuneAI) [ ![social icon](https://neptune.ai/wp-content/uploads/2022/08/icon-youtube.svg) ](https://www.youtube.com/channel/UCvOJU-ubyUqxGSDRN7xK4Ng) [ ![social icon](https://neptune.ai/wp-content/uploads/2023/01/icon-spotify.svg) ](https://open.spotify.com/show/4kGi82i4wTYgHbWmVMri5x) [ ![social icon](https://neptune.ai/wp-content/uploads/2023/01/icon-apple-podcast.svg) ](https://podcasts.apple.com/us/podcast/mlops-live/id1634179447)

[ Terms of Service ](/terms-of-service) [ Privacy Policy ](/privacy-policy) [ SLA ](/service-level-agreement)

Copyright © 2025 Neptune Labs. All rights reserved.

![](https://neptune.ai/wp-content/themes/neptune/img/footer-stars--414w.png)


