# documents

url: https://cloudmersive.com/blog
title: Blog - Cloudmersive APIs
scrape_date: 2025-03-14T16:01:05.295986
content_type: markdown
hash: 60d91ccff9b1e066662a81d38912205a7b14bcd16f8a86f17926f4ad14d39f71
content:
  * [ 1-415-234-7421](tel:1-415-234-7421)
  * [ Contact Us](https://portal.cloudmersive.com/contact-sales)
  * [ Login](https://portal.cloudmersive.com/login.aspx)



# [Cloudmersive.](/)®

  * [Products](/products)
  * [Solutions](/solutions)
  * [Pricing](/pricing)
  * [Docs](https://api.cloudmersive.com)
  * [Tools](/tools)
  * [![Search](/images/searchw.svg)](/search)
  * [Sign Up](https://portal.cloudmersive.com/signup)



![](/images/servers_header.png)

## [![](/images/map.svg) Blog](/blog)

Find out about the latest from Cloudmersive. [RSS Feed](/api/blogrss) | [Boost Macro, HTA, and VBS Security with the Cloudmersive Private Cloud Virus Scan API](/blog/Boost-Macro%2c-HTA%2c-and-VBS-Security-with-the-Cloudmersive-Private-Cloud-Virus-Scan-API)  
---  
3/10/2025 - Brian O'Neill  
  
  
Threat actors never stop retooling malicious payloads and improving the obfuscation tactics they use to disguise them. Staying ahead of this evolving threat landscape is the core of our mission at Cloudmersive. ![man holding cloud](https://docs.cloudmersive.com/api/image/blog-image/4ae2a4fa-b9d3-49c2-9b3a-c0f26e314613) In our latest Private Cloud Virus Scan API update, we’ve implemented enhanced detection for DOC and XLS (legacy Word and Excel) Macros, HTA threats, and VBA scripts. Understanding the danger these file types pose is an important step towards preventing devastating file-based cyberattacks. We've included some background information on each threat below.

#### Understanding DOC and XLS Macro Threats

Unlike their modern, Open Office XML (OOXML) file format equivalents – Word DOCX and Excel XLSX – legacy Word DOC and Excel XLS files are monolithic binary containers. That means ALL document components in DOC and XLS files – including text, formatting metadata, and embedded objects – are stored in a single structure. Relative to OOXML structure, which uses a series of compressed, human-readable XML files to store, cross-reference, and represent document content, this binary file structure is considered extremely opaque and consequently more difficult to investigate for threats. Obfuscating malicious Macros (and other threats) is easier in DOC and XLS file structure than in OOXML. DOCX and XLSX files are designed such that they can’t store Macros directly (Macro-enabled versions of these files are stored as DOCM and XLSM respectively). Conversely, binary DOC and XLS files can carry Macros embedded directly within the file with no external indication that this is the case. Accidental Macro execution is more likely in the legacy monolithic binary container format than it is in the modern OOXML. Rigorous investigation of DOC and XLS Macros is essential to avoid accidental Macro execution in legacy Office formats.

#### Understanding HTA Threats

HTML Application (HTA) files are a relatively uncommon file type, but their utility in Windows environments makes them an attractive attack vector. They can be used to build trusted, standalone applications on our desktop (with web-based UI), automate tasks via scripting, and develop interactive tools with local file access. ![press button launch executable](https://docs.cloudmersive.com/api/image/blog-image/0c9a60ca-2fb3-4e91-a399-8e99a2749a3b) HTA files are executable, and they run with full system privileges via mshta.exe (which allows HTML, JavaScript, and VBScript to run outside of a web browser without security restrictions). Their full system access makes them a formidable threat. Threat actors can deliver malware and/or initiate remote code execution attacks via HTA files, and they can just as easily initiate phishing attacks by tricking people into launching obfuscated HTA files from emails or downloads. It’s usually best to block mshta.exe via security policies to prevent HTA threats, but in scenarios where that’s impossible (i.e., mshta.exe is required for various apps and workflows), HTA files should be rigorously investigated for malicious content to avoid accidental execution.

#### Understanding VBS Script Threats

Like any script file, Visual Basic Script (VBS) files pose a direct, obvious threat and should be treated with the utmost caution. VBS files can be manipulated to initiate remote code execution attacks, modify files and registry settings, and execute harmful commands on our system. They can be embedded in email attachments, and they can be hidden within Office Macros or obfuscated with fake file extensions. Detecting obfuscated VBS files is critical to avoid VBS-based attacks. Script files are commonly saved with phony extensions, and clever social engineering tactics can easily lead to accidental execution or download. Obfuscated script files must be rigorously validated against the presented extension to determine that they do not conform with the standards of the expected file type (e.g., a VBS file saved with a PDF extension will not pass a PDF content validation check).

#### Private Cloud Virus Scan API Threat Detection

The Private Cloud Virus Scan API offers 360-degree content protection with scanning policies for viruses, malware, and a host of content threats, including Macros, scripts, unsafe archives, executables, and more. The Virus Scan API is continuously updated with the latest iterations of virus and malware signatures, and each content policy is continuously bolstered to account for new content threat iterations and obfuscation methods. It will defend against Macro, HTA, and VBS threats by investigating files “under the hood” for malicious code & rigorously validating file contents against the given file extension. For further information and/or expert advice on blocking Macro, HTA, and VBS threats with Cloudmersive, please feel free to [contact a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Enhanced ELF Executable Detection Added to Private Cloud Virus API](/blog/Enhanced-ELF-Executable-Detection-Added-to-Private-Cloud-Virus-API)  
2/25/2025 - Brian O'Neill  
  
  
![touching lock connected to important systems](https://docs.cloudmersive.com/api/image/blog-image/5392ab3d-08d4-42d2-a7d8-259767e0fcd0) We’re pleased to announce that enhanced ELF executable detection has been added to the Cloudmersive Private Cloud Virus API with the release of v10-1-2.

#### What are ELF executables?

The Executable and Linkable Format (ELF) is a special file format used to store compiled machine-readable files (binaries), reusable program code (libraries), and program memory snapshots (core dumps) in Linux and Unix-based systems. It’s similar to Windows-based Portable Execution files like Executables (EXE) and Dynamic-Link Libraries (DLL).

#### Why is it important to scan for ELF executable threats?

Before delving into the unique threats ELF files pose, it’s first important to note that ALL executable files pose a significant risk to their compatible operating system(s) and should be rigorously investigated for threats. Additionally, ALL executables can be quite easily obfuscated as other file types by simply renaming the file with a new extension (e.g., JPG), so it's important to always check files for hidden executable content. Because ELF files are the standard format for executables, shared libraries, and kernel modules on Linux and Unix-based systems, they’re a natural target for threat actors. Linux and Unix-based servers – including database servers, file servers, and application servers – rely on ELF files, and most Internet of Things (IoT) devices depend on ELF files for firmware and embedded software execution. Further, in much the same way as malicious EXE and DLL files on Windows systems, malicious ELF files are capable of granting unauthorized access or root privileges in Linux and Unix-based systems, which can lead to rapid & devastating attack escalation. Malicious ELF files can carry anything from known malware signatures to embedded payloads (i.e., shellcode, trojans, backdoors) and other malicious code injections. They can also carry links to malicious and/or vulnerable shared libraries.

#### Scan ELF executables with Cloudmersive

![Lock global content safety](https://docs.cloudmersive.com/api/image/blog-image/58fc8e74-4fe0-4c6a-9df6-fd17397bd41e) The Cloudmersive Virus Scan API combats ELF threats (and other executable threats) by scanning for malware and rigorously validating file contents beyond the file extension. The Virus Scan API can uncover malware signatures, scripts, and other suspicious content hidden within an ELF file, and it can also be configured to categorically block ELF files (and other executables) from upload/download workflows and cloud storage containers. For expert advice on scanning for ELF threats, please [reach out to a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Azure App Service Installation Now Available for Cloudmersive Reverse Proxy Server](/blog/Azure-App-Service-Installation-Now-Available-for-Cloudmersive-Reverse-Proxy-Server)  
2/11/2025 - Brian O'Neill  
  
  
![Cloud Server Graphic](https://docs.cloudmersive.com/api/image/blog-image/add3d883-4364-479c-9aa2-6de5ce662a98) In our most recent **Cloudmersive Reverse Proxy Server** update, we’ve included the option to install natively on **Azure App Service**.

#### What is Azure App Service?

**Azure App Service** is an HTTP-based service for hosting web applications, REST APIs, and mobile app backends.

#### How to Install a Cloudmersive Reverse Proxy Server on Azure App Service

Installing a **Cloudmersive Reverse Proxy Server** on **Azure App Service** is quick and easy. After we’ve navigated to the **Reverse Proxy Private Cloud Node** page in our Cloudmersive **Management Portal** , we’ll find the **Azure App Service** tab available on the far righthand side of our **Operating System** options. ![1 - find azure app option](https://docs.cloudmersive.com/api/image/blog-image/b0e267e9-f305-493a-a8f9-1fbffb9c3fb9) On the **Azure App Service** tab, we’ll find a list of instructions guiding us through **Azure App Service** installation. In the first step of our installation process, we’ll create a new **Azure App** in our **Azure Portal**. ![2 - click create on azure app service](https://docs.cloudmersive.com/api/image/original/f4f00c87-fde4-4a81-825a-1509ef010fd8) On our **Create Web App** page, we’ll select **Windows OS** with a **.NET 6 Runtime Stack**. ![3 - select runtime and OS](https://docs.cloudmersive.com/api/image/blog-image/e219a857-7519-4327-9a47-4910d9cfc765) After we create our **Azure App** , we’ll head to the **Configuration** page in our **App Settings** tab. Here, we’ll make sure we have a **32-bit Platform** selected. ![platform and 32 bit](https://docs.cloudmersive.com/api/image/blog-image/876780ff-77ab-40ff-8ac4-e9f04f04df22) From the **Configuration page** , we’ll navigate to the **Environment Variables** tab. We should see the option to head straight to **Environment Variables** tab at the top of the **Configuration** page. ![5 - link to enviro variables](https://docs.cloudmersive.com/api/image/blog-image/625e56e4-acce-441f-b68e-6898a7f035b0) On the **Environment Variables** tab, we’ll add a new application setting. ![7 - add variable](https://docs.cloudmersive.com/api/image/blog-image/93908d20-7b26-4f64-9579-a6fd63a7bdaa) We’ll **Name** this setting `WEBSITE_RUN_FROM_PACKAGE`, and we’ll select the **Value** `https://privatecloud.cloudmersive.com/download/CloudmersiveReverseProxyServer-v3-7-5.zip`. Please note – these strings can be found on the **Azure App Service** installation instructions in our **Reverse Proxy Private Cloud Node** page. ![7 - first name and value](https://docs.cloudmersive.com/api/image/original/735aeb31-174c-40fc-87c5-550a3201170c) After we apply this setting, we’ll create and apply one additional setting. We’ll **Name** this setting `CMPCAccessKey`, and we’ll select the **Value** `d6d0f1ff-3090-421a-a6e6-a7db3b92bc22`. ![8 - second name and value](https://docs.cloudmersive.com/api/image/blog-image/2aa6d02b-1446-4b05-93e2-447939835273) We’ll now click **Apply** on the **Environment Variables** tab to engage both settings. ![9 - apply settings](https://docs.cloudmersive.com/api/image/blog-image/a241fce0-ccc4-4c6e-a521-591cc703b666) To complete the process, we’ll head back to our **App Overview** page and **Restart** our **Azure App**. ![Restart App](https://docs.cloudmersive.com/api/image/blog-image/3cf948ca-f78b-44ab-b2d8-bdd1a90fd5cf) To confirm we completed all steps successfully, we can visit our **Azure App Service host URL**. ![demo host url](https://docs.cloudmersive.com/api/image/blog-image/22f59688-0522-4d4a-9078-36d243e7a1ac) We should see the **Cloudmersive Reverse Proxy Server** logo or our **Configured Site** (see demo site above) appear. And that’s all there is to it! For additional information or help resolving an issue in this process, please [reach out to a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Support for 7zip and Split RAR Scanning Added to Scan Azure Blob Cloud Storage API](/blog/Support-for-7zip-and-Split-RAR-Scanning-Added-to-Scan-Azure-Blob-Cloud-Storage-API)  
1/22/2025 - Brian O'Neill  
  
  
![Bad Archive](https://docs.cloudmersive.com/api/image/blog-image/e6531c44-62b0-4b0c-8f40-f3d0ab8e6deb) We’re pleased to announce that Scan Azure Blob Cloud Storage API functionality has been expanded to include support for 7zip and split RAR files. This expansion comes with the release of Cloudmersive Private Cloud Virus API V9-9-9.

#### What is a 7zip file?

7zip is an archive software which generates compressed files using the 7z file format. 7z is known for offering a particularly high compression ratio and support for multiple different formats, including ZIP, RAR, GZIP, TAR, and more.

#### What is a Split RAR file?

Split RAR files are RAR archives which have been divided into a series of separate, smaller RAR files (e.g., first_part.rar, second_part.rar, etc.). Split RAR files can be stored separately and/or transferred over networks in chunks. They’re typically used when platforms have significant file size limitations for upload & download, or for sharing files via email.

#### Why scan 7zip and split RAR files?

Like any archive format, 7zip and split RAR files can be used to obfuscate malware and other malicious files, such as executables, scripts, macro-enabled Office documents, and more. Cloudmersive 360-degree virus scanning will “lift the hood” to identify subfile threats hidden within 7zip and split RAR files and report malicious content in the API response body. For expert advice on dealing with 7zip and split RAR file threats, please [reach out to a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[New EditDocument API Added to Convert API Library](/blog/New-EditDocument-API-Added-to-Convert-API-Library)  
10/7/2024 - Brian O'Neill  
  
  
We're pleased to announce the addition of a brand-new API to the Cloudmersive Convert API Library. This addition comes with the release of [Convert API v6-6-2](https://cloudmersive.com/release-notes/Cloudmersive-Convert-API-v6-6-2-Released). ![holding pen working on files on computer](https://docs.cloudmersive.com/api/image/blog-image/6ae809fc-b5b4-4f4c-8a8f-abde5c057ff7)

### Introducing the `Begin Editing Chunked` API

The `Begin Editing Chunked` API is designed to enable loading large files as temporary editing URLs in environments where file sizes are capped or restricted. Large files loaded into memory with the `Begin Editing Chunked` API are broken into smaller pieces (i.e., “chunks”) to facilitate downstream `EditDocument` API operations.

### What is the `Begin Editing` API?

The standard `Begin Editing` API uploads documents to Cloudmersive and returns their contents as secure temporary URLs. These URLs facilitate high-speed file editing operations in subsequent `EditDocument` API calls. They're stored in-memory for security purposes, and they’re deleted from the cache after 30 minutes.

### How is the `Begin Editing` API used?

Once document contents are loaded as a temporary URL, that URL can be passed to other `EditDocument` APIs in the Convert API library for high-speed document processing workflows. For example, after loading a multi-page Word (DOCX) document with the `Begin Editing` API, you could pass the URL to the `Delete Pages` API using the `InputFileUrl` parameter instead of the `InputFileBytes` parameter. This would allow you to remove sections of the document in-memory. The `Delete Pages` API would return a modified version of the temporary editing URL which could be processed through additional `EditDocument` APIs in a multi-step file processing workflow. Once the chain of URL edits was finished, the process could be completed by calling the `Finish Editing` API, which downloads the modified document file bytes from the temporary editing URL and returns them as a string. The `Begin Editing Chunked` API ensures file request size restrictions do not hinder this process.

### Navigating large file request size constraints with Cloudmersive

Cloudmersive is proud to offer various APIs and flexible endpoint deployment solutions for working around file request size constraints in diverse customer environments. If you have any questions about file request size constraints in your environment, please do not hesitate to [contact a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Introducing Chunked Transfer Encoding Support for the Advanced File Scan API](/blog/Introducing-Chunked-Transfer-Encoding-Support-for-the-Advanced-File-Scan-API)  
9/30/2024 - Brian O'Neill  
  
  
![folders flying from filing cabinet - chunked transfer concept](https://docs.cloudmersive.com/api/image/blog-image/4f12bb26-c996-42c0-831d-e4052bb64f05) We’re pleased to announce that support for `chunked` transfer encoding has been added to the `Advanced File Scan` API. This addition comes with the release of [Cloudmersive Cloud Virus API v9-6-0](https://cloudmersive.com/release-notes/Cloudmersive-Cloud-Virus-API-v9-6-0). Support for `chunked` transfer encoding improves performance and memory management for the `Advanced File Scan` API in workflows involving large files.

### What is chunked transfer encoding, and what is it used for?

`Chunked` transfer encoding is a widely utilized web communication encoding method, originally introduced as part of the HTTP/1.1 protocol. The `chunked` encoding method makes it possible for servers and clients to keep up a persistent connection when partial file content is transferred in an HTTP request. This method is extremely useful for HTTP request scenarios involving large content data transfers or streams, especially when the exact length of content is unknown at the outset of the transmission.

### How does `chunked` transfer encoding work?

`Chunked` transfer encoding is enabled in an HTTP request when the `Transfer-Encoding` header is included and set to `chunked`. In a `chunked` transfer encoding request, a server sends content data to a client as a series of “chunks” rather than a single, monolithic entity. Instead of waiting for the entire transmission to arrive, the client can begin processing each chunk of content individually. Each chunk is preceded by information (hexadecimal) about its size, and when all content data is transferred, the end of that content is explicitly signaled to the client.

### Chunked transfer encoding with the `Advanced File Scan` API

Sending files to the `Advanced File Scan` API in chunks improves performance for requests involving large files. Thanks to `chunked` transfer encoding, file scans can begin when initial portions of a file are received and finish after the final portion of that file arrives.

### Scanning files with the `Advanced File Scan` API

The `Advanced File Scan` API offers 360-degree protection against viruses, malware, and custom content threats including macros, scripts, executables, and more. For more information on utilizing chunked transfer requests to scan large files with the `Advanced File Scan` API, please feel free to [contact a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Explore Recent Resize PDF API Updates](/blog/Explore-Recent-Resize-PDF-API-Updates)  
9/16/2024 - Brian O'Neill  
  
  
![documents face up on table near laptop - pdf resize concept](https://docs.cloudmersive.com/api/image/blog-image/3c45e7b1-e817-4948-9703-c556c2094c08) We’re excited to bring new `EditPdf` API updates to your attention! With the release of [Cloudmersive Convert API V-6-6-1](https://cloudmersive.com/release-notes/Cloudmersive-Convert-API-v6-6-1-Released), new `paperSize` parameter values have been added to the `Resize PDF` API. These values include the following new paper sizes:

  * `Letter`
  * `Legal`
  * `Tabloid`
  * `Ledger`
  * `B5`

Previously, `Resize PDF` values included `A7` through `A0`.

#### `Letter` size

Adding the `Letter` value to your `Resize PDF` request converts the existing paper size to 8.5 x 11 inches (216 x 279 mm). This is considered a standard paper size in the United States, and it’s commonly used for a wide range of documents.

#### `Legal` size

Requesting a page size conversion to `Legal` returns PDF pages that are 8.5 x 14 inches (216 x 356 mm) long. The `Legal` paper size is a version of `Letter` optimized for legal use-cases; it maintains identical horizontal proportions while accommodating extra page length.

#### `Tabloid` size

Utilizing the `Tabloid` value converts pages to 11 x 17 inch (279 x 432 mm) dimensions. As the name suggests, this page size is designed with publications (newspapers, magazines, etc.) in mind.

#### `Ledger` size

Converting PDF pages to `Ledger` size results in the same 11 x 17 inch (279 x 432 mm) dimensions as the `Tabloid` value. The only difference is that pages are returned in landscape orientation to facilitate the presentation of accounting materials (e.g., spreadsheets converted to PDF).

#### `B5` size

Specifying the `B5` PDF paper size returns 6.93 x 9.84 inch (176 x 250 mm) pages. This medium-sized option is commonly used for books and other stationery materials.

### Changing PDF paper sizes with Cloudmersive

`Resize PDF` requests are handled by passing PDF file bytes and a `paperSize` parameter string to the Convert API `EditPdf` endpoint in an `application/octet-stream` request. The `Resize PDF` API returns modified PDF file bytes as a string in its response. If you have any questions about utilizing new `Resize PDF` parameter values or leveraging `EditPdf` APIs as a whole, please feel free to [reach out to a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Extending SharePoint List Capabilities with Cloudmersive: Convert Excel Attachments to JSON in Power Automate](/blog/Extending-SharePoint-List-Capabilities-with-Cloudmersive%3a-Convert-Excel-Attachments-to-JSON-in-Power-Automate)  
8/28/2024 - Brian O'Neill  
  
  
These days, JSON format reigns supreme in most programming environments. That’s true even on no-code platforms like Power Automate, which conveniently offers built-in actions for parsing JSON data from API responses into individual dynamic content items. While it’s entirely possible to parse and leverage data in other formats – like CSV or even XLSX, for example – it’s far easier to perform dynamic data-related actions in Power Automate when we re-structure our data a JSON objects. In this installment of our SharePoint List Automation blog series, we’ll learn how to take full advantage of SharePoint Lists that gather standardized Excel spreadsheet attachments from team members. We’ll create a simple Power Automate flow that automatically converts our Excel list attachments to JSON objects in Power Automate, and we’ll use the **Cloudmersive Convert API** to handle our conversion. ![spreadsheet with pen and calculator](https://docs.cloudmersive.com/api/image/blog-image/d489efa4-e56c-4d05-ba10-02dcf28c3695)

#### What Can we Do with Excel Data Converted to JSON?

First and foremost, it's worth noting that we'll benefit significantly from converting Excel data to JSON in workflows designed to pass data to external web services. Most (if not all) web services want to consume simple JSON objects rather than complex file structures like XLSX, and that’s especially true when we’re dealing with cross-platform data exchange. If, for example, we're creating extremely advanced flows that send Excel datasets to NoSQL databases or share Excel information with external API endpoints via HTTP, we'll definitely want to use JSON. On a much less complex & more practical level, we’ll find JSON data dramatically simpler to use in most of our simple data transformation and automation workflows. Once we’ve converted spreadsheet data to JSON, we can parse that data into a custom schema, and we can use each individual item from that schema downstream in our flow. For example, we could automatically parse items from expense report spreadsheets and share those items in approval workflows involving important list stakeholders. In this example, we directly extend the utility of our SharePoint List and shave time off a critical business process without writing any code. ![JSOn data with blue lightning graphic](https://docs.cloudmersive.com/api/image/blog-image/3aa13cbb-7186-4e2a-8468-900f52b72182)

#### Converting Excel to JSON with the Cloudmersive Convert API

When it comes time to make our Excel to JSON conversion in this walkthrough, we’ll be using an iteration of the **Cloudmersive Convert API** labeled **Convert Excel XLSX to JSON conversion**. If you’ve followed other walkthroughs in our SharePoint List automation blog series, you’ll know we’ve talked ad nauseum about the two primary options available to take advantage of Cloudmersive APIs in Power Automate. We’ll go over that again now. The two options we have are **1)** taking advantage of a the **Cloudmersive Document Conversion** connector, which is published in the Power Automate connector database and uses a public cloud endpoint by default, and **2)** uploading the **Cloudmersive Convert API** to Power Automate as a custom connector, which allows us to set our own private or managed instance endpoint. If we’re a private or managed instance customer, we can [contact our sales representative](https://portal.cloudmersive.com/contact-sales) if we’re curious to learn more about setting up Cloudmersive APIs as custom connectors in Power Automate. The upload process is nice and straightforward; the API spec can be parsed via Power Automate’s OpenAPI URL option. If we’re new to using Cloudmersive connectors, we can [set up a free account](https://portal.cloudmersive.com/signup) to use the **Document Conversion** connector with a free API key. It’s important to note that we’ll need a premium Power Automate account to access Cloudmersive connectors, but with our free Cloudmersive API key, we’ll be able to make up to 800 API calls per month in perpetuity with zero additional commitments (our call total will reset each month). In this walkthrough, we’ll be using the published **Cloudmersive Document Conversion** connector in our example flow. This makes setting up the demonstration easier.

#### Step 1: Select the **Automated Cloud Flow** Option

We’ll start off this flow with the same initial steps we’ve used in previous SharePoint List automation walkthroughs. We’ll first select the option to create an **Automated cloud flow**. ![Image 1 - Select Automated Flow](https://docs.cloudmersive.com/api/image/original/74901e52-6b4d-40d1-852f-c014b9378053) And right after that, we’ll search for and select the SharePoint **When an item is created** trigger action. ![Image 2 - Select When an Item is Created](https://docs.cloudmersive.com/api/image/original/24d9d3bb-297c-4d10-9223-0766675990a3) To be clear, this action will only trigger our flow when _new_ items are added to our target list. If we want to trigger our flow when items are updated as well, we can use the **When an item is created or modified** trigger action instead. Keep in mind, however, that using this option will trigger an infinite loop error if we use our flow to update the original list item.

#### Step 2: Configure the Site Address and List Name

When we reach the flow diagram page, the first thing we’ll do is configure our trigger action. That means opening up our trigger action and selecting our SharePoint Site Address and List name from each respective dropdown. ![image 3 - configure trigger action json](https://docs.cloudmersive.com/api/image/original/e6aa4d20-1cbb-4d0f-b03a-9bff52ebfb3f) As a quick reminder, we’ll need to select these same two values for each subsequent SharePoint action in our flow.

#### Step 3: Get Excel Attachments and Attachment Contents

Now that our flow is primed to grab information from our target SharePoint List when new items are created, we can obtain information about Excel files attached to the trigger item, and we can get the file bytes we need for our JSON conversion directly after that. We’ll first add a SharePoint action called **Get attachments** into our flow. After we configure our list details, we’ll use add the list item ID (displayed as **ID** in the dynamic content window) from our trigger step into the **Id** parameter. ![image 4 - configure get attachments json](https://docs.cloudmersive.com/api/image/original/9bfb8137-4bbd-40c1-b046-4e51164c3ad3) Next, we’ll add a SharePoint action called **Get attachment content**. For this action, we’ll use both the list item ID from our trigger step and the file identifier ID from our **Get attachments** step to pull Excel file bytes into our flow. ![image 5 - configure get attachment contents json](https://docs.cloudmersive.com/api/image/original/9f10a1fa-171f-4e8f-8718-732facfe4552) We’ll notice that Power Automate automatically wraps the **Get attachment content** action in an **Apply to each** (displayed in-flow as **For each**) control to account for the possibility that multiple documents were attached to our triggering list item. If multiple Excel files are attached to an item in our target list, this portion of the flow will perform our conversion action for each separate attachment.

#### Step 4: Convert Excel Files Bytes to JSON

Within our **For each** control, we’ll add a new action and search for Cloudmersive connectors. As mentioned earlier, we’re looking for the published Cloudmersive **Document Conversion** connector in this example (if we were using the **Convert API** as a custom connector, we would select the **Custom** value from the **Runtime** dropdown instead). ![image 6 - find doc conversion connector json](https://docs.cloudmersive.com/api/image/original/1ac8d9cc-a24a-463b-829f-ff548cb226b9) Once we find this this connector, we can click **See more** to view the actions list. From here, we’ll use a control find search to locate the action labeled **Convert Excel XLSX to JSON conversion**. ![image 7 - doc conversion configuration json](https://docs.cloudmersive.com/api/image/original/00e8de11-43aa-4a4d-b924-8b32a690fb7a) When we find the correct action, we’ll select it, and we’ll then configure our connection by adding our API key if we haven’t used this connector before. If we’re using a custom connector, our connection should’ve been configured during the OpenAPI upload process. To configure our request, we’ll select **Attachment Content** from the **Get attachment content** step to satisfy our **Input File** parameter, and we’ll use the **DisplayName** value from our **Get attachments** step to satisfy our **File Name** parameter. ![image 8 - convert xlsx to json config](https://docs.cloudmersive.com/api/image/original/4893d74e-ba7d-4f16-a4e6-59df1ac7f8d1)  
At this point, we’ve officially converted our Excel data to a JSON array, and we can use that data the same way we would any JSON object in our Power Automate flows. In this example flow, we’ll next parse that data into a custom schema.

#### Step 5: Parse JSON

Using the **Parse JSON** action in Power Automate, we can pass the JSON array from our **Convert Excel XSLX to JSON** action into a custom schema. This schema allows us to use each field of our original Excel spreadsheet as dynamic content downstream in our flow. If, for example, our Excel list attachment contained basic expense report information like the following example: ![image 9 - example expense report](https://docs.cloudmersive.com/api/image/original/5ff95b7f-aa4b-4dbf-b6ad-75034b637b82) We would end up with a JSON array structured like the below example:
    
    
    [
      {
        "Date": "8/1/2024",
        "Category": "Travel",
        "Description": "Flight to NYC",
        "Amount": "350",
        "Currency": "USD"
      },
      {
        "Date": "8/2/2024",
        "Category": "Meals",
        "Description": "Dinner with client",
        "Amount": "75.5",
        "Currency": "USD"
      },
      {
        "Date": "8/3/2024",
        "Category": "Accommodation",
        "Description": "Hotel stay",
        "Amount": "450",
        "Currency": "USD"
      },
      {
        "Date": "8/4/2024",
        "Category": "Transport",
        "Description": "Taxi to airport",
        "Amount": "45",
        "Currency": "USD"
      },
      {
        "Date": "8/5/2024",
        "Category": "Office Supplies",
        "Description": "Printer paper",
        "Amount": "25",
        "Currency": "USD"
      }
    ]
    

And we could parse that array into a schema like the below example:
    
    
    {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "Date": {
            "type": "string",
            "description": "Date of the expense in string format"
          },
          "Category": {
            "type": "string",
            "enum": ["Travel", "Meals", "Accommodation", "Transport", "Office Supplies"],
            "description": "Category of the expense"
          },
          "Description": {
            "type": "string",
            "description": "A brief description of the expense"
          },
          "Amount": {
            "type": "string",
            "description": "The amount spent as a string"
          },
          "Currency": {
            "type": "string",
            "enum": ["USD"],
            "description": "The currency used for the expense"
          }
        },
        "required": ["Date", "Category", "Description", "Amount", "Currency"]
      }
    }
    

All told, this would appear in our flow like the below screenshot: ![image 10 - parse json schema](https://docs.cloudmersive.com/api/image/original/772823b3-aa5a-4ff1-9f66-48af4f2bada1) We could, for example, use our parsed JSON information to create approvals for each individual item on the expense report. Thanks to our **For each** control, adding a single approval action would send multiple separate approvals to stakeholders for each individual item in the original spreadsheet. We could then implement a feedback loop that shared approval information directly with the employee who originally attached the spreadsheet to the SharePoint List. This is just one example of how we can leverage our converted Excel data in a simple flow. Once we’ve converted Excel data to JSON, the possibilities are endless!

#### Conclusion

In this walkthrough, we’ve learned how to leverage the **Cloudmersive Convert API** in Power Automate to automatically convert Excel attachments to JSON when they’re added to a specific SharePoint List. If you have any questions about using Cloudmersive APIs in your Power Automate flows, please feel free to [reach out to a sales representative](https://portal.cloudmersive.com/contact-sales) – they’ll be happy to answer any questions you may have and help create POC’s as needed.  
  
  
  
[Automatically Watermark PDFs Attached to a SharePoint List in Power Automate](/blog/Automatically-Watermark-PDFs-Attached-to-a-SharePoint-List-in-Power-Automate)  
8/20/2024 - Brian O'Neill  
  
  
We’re back again this week with another Power Automate walkthrough! This week, we’re going to leverage yet another Cloudmersive Convert API action in a SharePoint List automation scenario. Specifically, we’re going to learn how to automatically apply our own custom text watermarks to PDF documents when they’re attached to a target SharePoint List. If watermarking is an essential part of our team’s document compliance workflow, this simple automation might save us a ton of time and effort in the long run. If you'd prefer to watch a video version of this walkthrough, please [follow this link](https://cloudmersive.com/videos/How-to-Watermark-PDFs-Attached-to-a-SharePoint-List-using-Power-Automate). ![floating document over hands on keyboard](https://docs.cloudmersive.com/api/image/blog-image/9de3196a-ae3d-4dcf-886c-a14ca62088ae)

#### What is a Watermark?

Watermarks are obstructive labels that protect content from unauthorized use, copying, or distribution. They’re typically semi-transparent text, logos, or patterns overlaid on a document, and they’re commonly used across a wide range of industries. In Financial Services, for example, watermarks are frequently applied to PDFs (and other documents) to indicate the degree of confidentiality associated with any given document and eliminate any uncertainty around who’s privileged to view that content. Government agencies also lean on watermarks to ensure document authenticity and create clear trails for audits and other compliance checks.

#### Watermarking with the Cloudmersive Convert API

The iteration of the Convert API we’ll use today is appropriately labeled **Add a text watermark to a PDF**. This API allows us to customize the font name, size, color, and transparency of our PDF text watermark. To be clear, this API is ONLY for creating text watermarks. Before we get to our walkthrough, we’ll first discuss our options for using this API in Power Automate. As a quick reminder, we can take advantage of Cloudmersive Convert API actions in Power Automate in two different ways: we can either use published Cloudmersive connectors - such as the **Document Conversion** connector, **File Processing** connector and **PDF** connector - or we can upload the Convert API to Power Automate as a custom connector. If we’re a current enterprise customer leveraging Cloudmersive APIs with a private or managed instance endpoint, using a custom connector route will give us the flexibility to make API calls through Power Automate against our own endpoint (if you have any questions about configuring this correctly, please feel free to [contact your sales representative](https://portal.cloudmersive.com/contact-sales)). If we’re new to using Cloudmersive connectors, we can easily leverage the published suite of Cloudmersive connectors with a public cloud connection by creating a free account (on the Cloudmersive [website](https://cloudmersive.com/)) and getting a free API key. Free API keys allow a limit of 800 API calls per month with zero commitments; that total resets each month in perpetuity, and if 800 API calls gets the job done, we’ll never have to scale up. ![Compliance technology concept](https://docs.cloudmersive.com/api/image/blog-image/bae845e2-4141-4db8-915f-bf44379c6651)

#### Step 1: Create an **Automated Cloud Flow**

In this walkthrough, we’re creating a flow that triggers when new items with PDF attachments are added to a specific SharePoint list. That means we’ll start by selecting the **Automated cloud flow** option. ![Image 1 - Select Automated Flow](https://docs.cloudmersive.com/api/image/original/74901e52-6b4d-40d1-852f-c014b9378053) From here, we’ll search for and select the SharePoint **When an item is created** trigger action. This action kicks into gear when new brand-new items are added to a SharePoint List. ![Image 2 - Select When an Item is Created](https://docs.cloudmersive.com/api/image/original/24d9d3bb-297c-4d10-9223-0766675990a3) As a reminder, we could also use the **When an item is created or modified** trigger action if we’re interested in watermarking PDFs attached to existing list items, but it’s important to remember that using this action can result in infinite loops if we don’t structure our flow properly.

#### Step 2: Configuring Initial SharePoint Actions

We’ll kick off designing our flow by opening our trigger action, configuring our SharePoint connection with our account details, and selecting our Site Address and List name from each respective dropdown. ![Image 3 - Configure Trigger Action](https://docs.cloudmersive.com/api/image/original/a2e2fb8f-2696-4b63-8832-daf5ac67fb2f) We’ll need to select the same Site Address and List Name in each subsequent SharePoint action we implement in our flow. That includes the next action we’ll incorporate, which is the SharePoint **Get attachments** action. This action uses the **ID** (List item ID) value from our trigger action to retrieve an array of information about each file attached to our target list item. We’ll eventually use that information to get the attachment content. ![Image 4 - Configure Get Attachments](https://docs.cloudmersive.com/api/image/original/d5113470-133e-4a6b-8f71-651768c69df4)

#### Step 3: Limit File Type to PDF

Since the API we’re building this flow around only works on PDF documents, we’ll need to implement a basic **Control** to limit our file selection to PDF. Otherwise, our flow runs might fail more than we’d like and inundate us with error messages. We’ll take care of this by implementing a basic **Condition**. We’ll configure our condition to make sure the **DisplayName** value (retrieved by the **Get attachments** action) contains the string “.pdf”. ![Image 5 - condition limiting file type](https://docs.cloudmersive.com/api/image/original/7d31102c-869d-465b-b984-a1b6971f6b88) We’ll notice that selecting the **DisplayName** value causes Power Automate to automatically wrap our **Condition** in a **For each** (also referred to as **Apply to each**) control. This is to account for the fact that multiple unique files might be attached to the same list item.

#### Step 4: Get File Content for Each Attachment

Within the **True** branch of our **Condition** , we’ll now add in our SharePoint **Get attachment content** action. We’ll use the original list item **ID** from our trigger action and the File Identifier **Id** from our **Get attachments** action to retrieve content from each list item attachment. ![Image 6 - configure get attachment content action](https://docs.cloudmersive.com/api/image/original/35dff0e8-a0ea-47ee-80b1-9abd4439e51c)

#### Step 5: Create a Custom Watermark

Now that we’ve brought PDF attachment content into our flow, we can finally implement our custom watermarking action via the Cloudmersive Convert API. For the purposes of this walkthrough, we’ll look for this action in a published Cloudmersive connector and make our calls against a public cloud endpoint; as a reminder, however, we can also use a custom Convert API connector to set our own private or managed instance endpoint. We’ll find the Convert API action we’re looking for in the **Cloudmersive PDF** connector. To bring up the list of published Cloudmersive connectors and find the **Cloudmersive PDF** connector, we just need to add a new action and type “Cloudmersive” into the search bar. ![Image 7 - find pdf conversion connector](https://docs.cloudmersive.com/api/image/original/bd2deb34-52f7-41de-8077-8cfcceb24892) When we click **See more** next to the **Cloudmersive PDF** connector, we’ll find the action we’re looking for - **Add a text watermark to a PDF** \- atop the alphabetically organized list. ![Image 8 - find text watermark action](https://docs.cloudmersive.com/api/image/original/a9eacee5-6ff2-45c3-8b19-3f081bfcaea7) Once we select the **Add a text watermark to a PDF** action, we’ll need to configure our Cloudmersive connection with our API key if we haven’t already. After that, we can use the **Attachment Content** value from our **Get attachment content** action to provide PDF file bytes for the operation, and we can use the **DisplayName** value to satisfy the file name variable. Next, we’ll enter our watermark text string into the variable labeled **Watermark Text To Add To The PDF**. We can then click **Show all** to view the advanced parameters, and from here we can customize the font family, size, color, and transparency of our watermark. If we leave all these values empty, we’ll end up with a size 150, red, Times New Roman watermark at 0.5 transparency. If we’re comfortable with default values, we don’t have to enter any information here. Towards the end of this article, we’ll see what that exact watermark looks like on an example PDF. ![Image 9 - configure watermark action](https://docs.cloudmersive.com/api/image/original/dd04507d-533b-4512-85dc-019b02ba59a0)

#### Step 6: Attach the Watermarked PDF to the Original SharePoint List Item

Now that we’ve completed our custom watermarking step, we can add one final action into our flow that attaches the new watermarked PDF to the original list item. We’ll use the SharePoint **Add attachment** action for this purpose. We’ll specify the original list item by once again supplying the list item **ID** , and we’ll satisfy the File Content variable using **OutputContent** from our watermarking action. We can name our file using any naming convention that makes sense to use; in this example, we will simply type “Watermarked” next to the original attachment **DisplayName** to easily distinguish our modified PDF content from the original document. ![Image 10 - configure add attachment action](https://docs.cloudmersive.com/api/image/original/e88069ba-f695-42fa-a135-b2b791d58375)

#### Step 7: Test the Flow and Generate a Watermarked PDF

We’ll now click **Save** in the upper right corner of the page, and once our flow successfully saves, we’ll click **Test** directly after. We’ll then select the option to trigger our flow **Manually** , and we’ll click **Test** once again. Power Automate will now wait for our flow’s trigger action to occur, which means we’ll need to attach a new PDF document to the target SharePoint List. When our flow finishes running, we’ll find the watermarked version of our PDF attached to the original list item (note that we may have to refresh our SharePoint List page for the new PDF to populate). ![Image 11 - list item with both pdfs attached](https://docs.cloudmersive.com/api/image/original/2b109df7-66dd-4b9a-a495-ae567139f405) We can now open our watermarked document and review the output of our operation. If we used the default API settings and entered a text string of similar length to “Watermark”, our result should look similar to the below example: ![Image 12 - watermarked PDF example default settings](https://docs.cloudmersive.com/api/image/original/dc1e99d1-a938-463d-bb6a-9a51ad99756b)

#### Conclusion

And that’s all there is to it - we've reached the end of our walkthrough! We can now easily add custom watermarks to PDFs automatically when they're attached to a relevant SharePoint List. If you have any questions about automating SharePoint List actions with Cloudmersive APIs, please feel free to [reach out to a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Automate SharePoint List Actions with Cloudmersive Connectors](/blog/Automate-SharePoint-List-Actions-with-Cloudmersive-Connectors)  
8/14/2024 - Brian O'Neill  
  
  


#### SharePoint List Automation

SharePoint Lists are an increasingly popular digital collaboration feature available in the Microsoft ecosystem. Lists can be exhaustively customized for anything from issue and expense tracking to company-wide surveys and announcements, and the addition of new list features - such as the recent incorporation of form entry URLs for SharePoint lists, for example - is constantly expanding that functionality. ![Business automation woman white and gray on computer](https://docs.cloudmersive.com/api/image/blog-image/396b3090-7923-4202-8818-0e184b78dc33) Thanks to Power Automate, the functionality of SharePoint Lists can be supported and expanded even further – all without writing a line of code. With Power Automate’s extensive suite of built-in Microsoft ecosystem connectors, we can trigger all kinds of value-add workflows when actions are taken in our SharePoint Lists. One common example of this is triggering the **Office 365 Outlook** connector to automatically email list attachments or new work assignments to list members; another is setting up automated stakeholder approval workflows via the **Approvals** connector.

#### Enhanced Automation with Cloudmersive APIs

Using Cloudmersive APIs as connectors in Power Automate, we can advance the capabilities of our lists even further. We can automatically validate, convert, and analyze the contents of documents attached to our SharePoint lists, and we can even implement advanced virus and malware scans before files are distributed to sensitive locations inside and outside of our network. As always, flexibility is the name of the game with Cloudmersive APIs: we can use Cloudmersive APIs as built-in Power Automate connectors to make low-scale public cloud endpoint calls, or we can create custom connectors that leverage our own Private and Managed Instance endpoints for hyper scalability and enhanced data security.

#### SharePoint List Automation with Cloudmersive: Blog Series

In our SharePoint List Automation blog series, we’ll walk through some value-add Power Automate flows that leverage Cloudmersive APIs to improve SharePoint List capabilities. This week, we’ll start with a flow that automatically converts all SharePoint List Item attachments to PDF and attaches the new document(s) back to the original list item. If you'd prefer to watch a video version of this walkthrough, [please follow this link](https://cloudmersive.com/videos/How-to-Convert-SharePoint-List-Item-Attachments-to-PDF-in-Power-Automate). ![automation graphic concept](https://docs.cloudmersive.com/api/image/blog-image/96a21013-a3b1-41b1-94e6-2c79cf3fd5cd)

#### Convert SharePoint List Item Attachments to PDF using the Cloudmersive Convert API

The Cloudmersive Convert API offers several unique actions for handling PDF conversions. As mentioned earlier, we can access those actions in Power Automate through an already-published connector that makes calls to a public cloud endpoint, or through a custom connector that makes calls to our Enterprise plan’s custom deployed endpoint. If we’re having trouble deciding which option is best for us, we can [contact](https://portal.cloudmersive.com/contact-sales) a sales representative and discuss the scope of our project with them. The specific Convert API iteration we’ll want to use in this case is the **Convert Document to PDF** API. There are several Convert API iterations designed to handle specific format conversions to PDF (e.g., DOCX to PDF or XLSX to PDF), but the **Convert Document to PDF** API is designed to autodetect a wide range of unique input file types and make high-fidelity conversions without any additional intervention. Using this broadly scoped API with our SharePoint List makes a lot of sense; we’ll be able to automatically convert anything from spreadsheets to written documents, images, and presentations without implementing extra steps to distinguish specific file types.

#### 1: Selecting the Right Type of Flow

Our goal is to design a flow that automatically triggers when new items are added to a target SharePoint List. That means we should start by creating an **Automated cloud flow**. Automated flows trigger when a designated event occurs, and in this case the event will be the addition of a new item to a specific list. ![Image 1 - Select Automated Flow](https://docs.cloudmersive.com/api/image/original/74901e52-6b4d-40d1-852f-c014b9378053)

#### 2: Selecting the Right Trigger

Before we head to the flow diagram page, we’ll want to pick a trigger action that describes our list event. In this case, we’ll select the SharePoint **When an item is created** trigger. ![Image 2 - Select When an Item is Created](https://docs.cloudmersive.com/api/image/original/24d9d3bb-297c-4d10-9223-0766675990a3) We could also use the **When an item is created or modified** trigger for this flow to account for scenarios where new files are attached to existing list items. If we go that route, however, we’ll need to implement various controls in our flow to prevent an infinite loop (i.e., a flow that re-triggers itself by attaching the converted PDF back to the original list item).

#### Step 3: Configure the SharePoint Site Address and List Name

If we haven’t yet configured our SharePoint connection in Power Automate, now’s the time to do it. For our initial **When an item is created** trigger action – and for every SharePoint action we use subsequently in this flow – we’ll need to select the SharePoint Site Address and SharePoint List name associated with the list we’re creating the flow for. We can see those selections below (we won’t cover this part again in our walkthrough). ![Image 3 - Configure Trigger Action](https://docs.cloudmersive.com/api/image/original/a2e2fb8f-2696-4b63-8832-daf5ac67fb2f)

#### Step 4: Get Attachments and Attachment Content

The next two actions we’ll use in our flow will **1)** retrieve the attachment information from our list item and **2)** retrieve the attachment file bytes. We can find both actions in the SharePoint connector actions list; the first is labeled **Get attachments** , and the second is labeled **Get attachment content**. In our **Get attachments** action, we’ll satisfy the **Id** field using the list item ID from our trigger action. ![Image 4 - Configure Get Attachments](https://docs.cloudmersive.com/api/image/original/d5113470-133e-4a6b-8f71-651768c69df4) In our **Get attachment content** action, we’ll once again use the list item ID from our trigger action, and we’ll then add the **File identifier** value (expressed as **Id**) retrieved by our prior **Get attachments** action. ![Image 5 - Configure Get Attachment Content](https://docs.cloudmersive.com/api/image/original/4156c4ba-28fe-481d-b77a-bcc609520010) We’ll notice that selecting the **Id** value automatically wraps our **Get attachment content** action in a **For each** (also known as **Apply to each**) control. Power Automate does this automatically to account for the possibility that multiple files are attached to a single list item.

#### Step 5: Convert Attachment Content to PDF

We’ll now incorporate our Cloudmersive Convert API action to convert our list item attachment contents to PDF. In the context of this walkthrough, we'll look for this action on the published Cloudmersive Document Conversion connector actions list; remember, however, that we could also use this action with a custom endpoint by uploading the Convert API to Power Automate as a custom connector. To reach the **Cloudmersive Document Conversion** connector actions list, we’ll add a new action within our **For Each** control, then type "Cloudmersive" into the connector search bar, and finally click **See more** next to the **Cloudmersive Document Conversion** connector with the green logo. ![Image 6 - Find Doc Conversion Connector](https://docs.cloudmersive.com/api/image/original/1147775f-6fa2-4d9a-b758-fa0de7bc57f9) From here, we’ll search for an action called **Convert Document to PDF** , and we’ll select it once we find it. ![Image 7 - Find PDF Conversion Action](https://docs.cloudmersive.com/api/image/original/b750d33f-f7e4-4b98-b0b2-c55288b07850) If we haven’t used this connector before, we’ll need to name our connection and provide an API key to configure our connection. To take care of this, we can simply copy and paste an API key from our Cloudmersive account portal, and we can then name our connection whatever we would like. If we don't yet have a Cloudmersive account, we can easily sign up for a free account to get started. Free accounts provide a free API key which allows a limit of 800 API calls per month (resetting each month in perpetuity) with zero additional commitments. To configure our request variables, we’ll use **Attachment Content** retrieved in the previous action, and we’ll enter the **DisplayName** value from our **Get attachments** step (note that the name we use doesn’t actually matter). ![Image 8 - Configure PDF conversion action](https://docs.cloudmersive.com/api/image/original/a6328002-414b-4467-89cd-1bcf4a1a551d)

#### Step 6: Attach the PDF to the SharePoint List Item

In the final step of our flow, we’ll use the SharePoint **Add attachment** action to attach our new PDF document to the same list item we retrieved the original content from. ![Image 9 - Configure Add Attachment](https://docs.cloudmersive.com/api/image/original/282954ab-047a-4e23-8067-a06e73362263) After we name our file, we will use **OutputContent** from our PDF conversion step to define the contents of our new list item attachment. We will also use the original list item ID to define our attachment location once again.

#### Step 7: Test the Flow

To test our flow, we will click **save** and then **test** in the upper right-hand corner of the page. We will then select the option to trigger our flow **Manually** and click **Test** once again. This will prompt us to test our flow by performing the starting action, which means creating a new list item with a file attachment. The **Convert Document to PDF** action supports all major Office formats, HTML, and more than 100 unique image formats. ![Image 10 - Final Flow Run](https://docs.cloudmersive.com/api/image/original/0116c9e6-15e6-47ab-b6c9-1014650d4aac) When our flow finishes running, we will find a PDF version of our original document attached to the same list item. ![Image 11 - PDF attached to list](https://docs.cloudmersive.com/api/image/original/aec4f005-ff78-427a-9d76-e47f9898a9a6) And that’s all there is to it! If you have any questions about designing flows with Cloudmersive APIs in Power Automate, please feel free to [contact a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Advanced Compare DOCX API Now Available for Private Cloud](/blog/Advanced-Compare-DOCX-API-Now-Available-for-Private-Cloud)  
6/21/2024 - Brian O'Neill  
  
  
![legal team reviewing document comparison docx](https://docs.cloudmersive.com/api/image/blog-image/80a57e3d-bdca-47b3-963f-9a0331cbd355) We’re pleased to announce the release of new, advanced Compare DOCX API capabilities in our latest [Private Cloud Convert API update (v-6-5-1)](https://cloudmersive.com/release-notes/Cloudmersive-Private-Cloud-Convert-API-v6-5-1-Released). An `Advanced` comparisonMode is now available for Private Cloud customers. This feature expands and enhances Compare DOCX API functionality. To learn more about this latest update, please feel free to reach out to your sales representative directly, or [contact any member of our team](https://portal.cloudmersive.com/contact-sales).

### Compare DOCX API Overview

The Compare DOCX API makes it easy for enterprises to automate comparisons between multiple versions of a DOCX document. Legal contract reviews, for example, can be greatly enhanced by the ability to compare final iterations of a contract document to the standard contract document it originated from.

### Calling the Compare DOCX API Without Code

The Compare DOCX API can be used without any code changes in a Power Automate flow. Further, by uploading the Cloudmersive Convert API to Power Automate as a custom connector, enterprises can make Compare DOCX API calls against a Private Cloud endpoint. This simply requires replacing the default Cloudmersive endpoint (api.cloudmersive.com) with your private cloud endpoint.

### Private Cloud Overview

Private Cloud customers benefit from high availability, single-tenant infrastructure. Private Cloud deployments can take place on global Cloudmersive infrastructure, global public cloud infrastructure (AWS, Azure, GCP, etc.), and on-premises infrastructure (i.e., client data centers). To learn more about Cloudmersive Private Cloud deployments, please feel free to [contact a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Optimize QR Barcode Scanning with Deep Learning AI](/blog/Optimize-QR-Barcode-Scanning-with-Deep-Learning-AI)  
6/12/2024 - Brian O'Neill  
  
  
![QR codes blurry and clear](https://docs.cloudmersive.com/api/image/blog-image/8d130df9-615f-4674-8a17-c9cc72a0ebea) We’re excited to announce a brand-new addition to the Cloudmersive Barcode API! The **Advanced QR Barcode Scanning API** is optimized to recognize low resolution or blurry QR barcode images using deep learning artificial intelligence. This API supports JPG, PNG, and PDF image formats, and it’s capable of processing more than one QR barcode image in a single operation.

### Effects of Low-Quality Barcode Images

Low-quality QR barcode images can significantly hamper the efficacy of QR barcode scanning applications. Common image processing operations such as image compression, image scaling, and image format conversion can all contribute to low-quality and unreadable QR barcode images. Each module of a QR barcode represents an important piece of the original encoded text data (e.g., an encoded URL). If any of those modules are left pixelated or distorted, they may fail to represent the original encoded text contents. Additionally, like most barcode images, QR barcodes rely heavily on a clear distinction between black and white areas in the barcode image. In a blurry barcode image, boundaries between the black and white areas of a QR barcode can become difficult to distinguish, making it impossible for line detections to occur accurately in the barcode reading process. The **Advanced QR Barcode Scanning API** automatically corrects blurry or low resolution QR barcode images to accurately read and extract encoded text.

### Learn More

Interested in learning more about the Cloudmersive Advanced QR Barcode Scan API? Please feel free to reach out to a member of our [sales team](https://portal.cloudmersive.com/contact-sales)!  
  
  
  
[Testing Cloudmersive APIs with cURL](/blog/Testing-Cloudmersive-APIs-with-cURL)  
6/5/2024 - Brian O'Neill  
  
  
![API graphic](https://docs.cloudmersive.com/api/image/blog-image/f1d076b8-9d1e-4403-bfee-f3defd855dff) Whether we’re testing API calls to a Cloudmersive Public Cloud, Private Cloud, or Managed Instance endpoint, cURL offers a convenient solution. Below, we’ll learn more about what cURL is and how we can use it to test Cloudmersive APIs.

#### What is cURL?

cURL stands for “Client URL”. At a high level, cURL is a command-line utility that allows us to send and receive data using URL syntax. We can use cURL to transfer data using a wide range of different protocols, including HTTP, HTTPS, FTP, FTPS, SCP, SFTP, LDAP, and more.

#### What is cURL API Testing?

When we test an API with cURL commands, we send HTTP requests directly from our terminal or command prompt to an API endpoint. This lightweight testing method is used to confirm the APIs our applications rely on are modifying and/or returning data the way we expect them to. It’s worth noting that using cURL commands to test Cloudmersive APIs is extremely similar to using Postman for the same purpose. The major difference is that Postman offers a user-friendly GUI with plenty of advanced features. If you’d prefer to go that route, feel free to [check out a different post on that subject](https://api.cloudmersive.com/postman.asp).

#### Example: Testing the Cloudmersive Basic Virus Scan API with cURL Commands

Preformatted cURL command examples are available to all Cloudmersive customers through the [Management Center](https://portal.cloudmersive.com/default) in the **Cloudmersive Account Portal**. To test any Cloudmersive API with cURL commands, we first need to navigate to the [Docs & Examples](https://portal.cloudmersive.com/documentation) **Documentation** page from the **Management Center**. Here, we’ll find code examples we can use to structure Cloudmersive API calls in a variety of common programming languages.

##### Find cURL Command Examples

First, let’s select the cURL option from the **Programming Language** list. Right after that, we can select the Virus Scan API option from the **API** list. This will bring up cURL command examples for each iteration of the Cloudmersive Virus Scan API in **Step 2 – Get Started with Example Code**.

##### Find the Basic Virus Scan API Iteration

If we type “Scan a file for viruses” in the **Search APIs** field, we’ll bring up cURL examples for the Basic Virus Scan API iteration right away. Alternatively, we can skip the search and simply scroll this list until we find that option.

##### Format the Examples for our OS

These are the cURL command examples we should see:
    
    
    curl --location --request POST 'https://api.cloudmersive.com/virus/scan/file' \
    --header 'Content-Type: multipart/form-data' \
    --header 'Apikey: YOUR-API-KEY-HERE' \
    --form 'inputFile=@"/path/to/file"'
    

It’s important to note that the above default examples are formatted for Unix-like command terminals. If we want to reformat our commands for a Windows command prompt, we’ll need to replace backslashes with carets, and we’ll need to replace single quotes with double quotes. Below, we’ll find the same examples formatted correctly for a Windows command prompt:
    
    
    curl --location --request POST "https://api.cloudmersive.com/virus/scan/file" ^
    --header "Content-Type: multipart/form-data" ^
    --header "Apikey: YOUR-API-KEY-HERE" ^
    --form "inputFile=@\"C:\path\to\file\""
    
    

Now we’ll break down each line of our example commands and determine what information we need to complete our own request.

##### Endpoint URL
    
    
    curl --location --request POST 'https://api.cloudmersive.com/virus/scan/file' \
    

The URL shown above is the default Cloudmersive Public Cloud endpoint. Since we’re testing the Basic Virus Scan API in this example, we’re making a POST request to that endpoint. If we want to make POST request to our Managed Instance or Private Cloud endpoint instead, we’ll need to replace this URL with our own endpoint URL. Depending on our subscription, we can also test a wide range of Public Cloud endpoints. We’ll find these by navigating to the [API Endpoint](https://portal.cloudmersive.com/endpoints.aspx) page from our **Management Center**.

##### Request Headers
    
    
    --header 'Content-Type: multipart/form-data' \
    --header 'Apikey: YOUR-API-KEY-HERE' \
    

Our cURL request has two separate headers. The first header defines the HTTP content type (in this case, multipart/form-data), and the second header captures our API key to authorize the request. We can copy an API key from the [API Keys](https://portal.cloudmersive.com/keys) page in the Cloudmersive Account Portal and paste that into our second header, replacing the `YOUR-API-KEY-HERE` placeholder text.

##### Request Form
    
    
    --form 'inputFile=@"/path/to/file"'
    

The form line captures the data we’re sending to the API endpoint in our POST request. We can replace the `”/path/to/file”` example with the path to whichever file we’ve decided to use in our test.

#### Carrying Out our cURL Test

Once we’ve properly formatted and filled out our cURL request, we can paste the cURL commands into our terminal or command prompt. This should execute our cURL request right away. A successful Basic Virus Scan API test with a clean file will return the following response:
    
    
    {"CleanResult":true,"FoundViruses":null}
    

And that’s all there is to it! We can easily use cURL examples to test any API on the Cloudmersive API platform.

#### Need Additional Help?

If you have any additional questions about testing Cloudmersive APIs with cURL command examples, please feel free to [reach out to a member of our support team](https://portal.cloudmersive.com/contact-support).  
  
  
  
[Announcing the New and Improved Compare DOCX API](/blog/Announcing-the-New-and-Improved-Compare-DOCX-API)  
4/29/2024 - Brian O'Neill  
  
  
We’re pleased to announce that we’ve added a brand-new feature to the Compare DOCX iteration of the Cloudmersive Private Cloud Convert API. It’s now possible to compare the header and footer sections of DOCX documents by setting a `headersAndFooters` parameter to `true`. The Compare DOCX API helps streamline the difficult and tedious process of identifying differences between similar Word documents. ![stack of documents with clamps](https://docs.cloudmersive.com/api/image/blog-image/f3e1e00a-4347-4928-ae69-5b0b76f96f01) Cloudmersive Private Cloud Convert API customers can leverage the Compare DOCX API to identify discrepancies between two different versions of an important document (e.g., legal contracts and invoices). Such documents tend to change hands multiple times and undergo several rounds of edits during ongoing negotiations with clients and customers. The addition of the `headersAndFooters` parameter allows for a more comprehensive document comparison. For more information on the Cloudmersive Private Cloud Convert API, please do not hesitate to [reach out to a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[CVE Spotlight: Learning from CVE-2023-5474 PDF Heap Buffer Overflow Vulnerability](/blog/CVE-Spotlight%3a-Learning-from-CVE-2023-5474-PDF-Heap-Buffer-Overflow-Vulnerability)  
4/11/2024 - Brian O'Neill  
  
  
![digital development graphic](https://docs.cloudmersive.com/api/image/blog-image/a4185d42-06f1-4923-a5b8-9dcb16cc417c) In October 2023, multiple PDF-related vulnerabilities were discovered in Google Chrome that could lead to [arbitrary code execution](https://cloudmersive.com/article/What-is-Arbitrary-Code-Execution%3f) attacks. In the **CVE-2023-5474** example (which earned an exceptionally high base severity score of 8.8 in the National Vulnerability Database), a remote attacker could trigger a [heap buffer overflow](https://cloudmersive.com/article/What-is-a-Heap-Based-Buffer-Overflow-Vulnerability%3f) in a victim’s web browser by convincing them to interact with certain features of a specially crafted malicious PDF file.

#### Understanding the Severity of Heap Buffer Overflows

Heap buffer overflows occur when a program writes more data to a heap memory buffer than the buffer is supposed to handle. This memory "overflow" can cause data stored in adjacent memory buffers to become corrupted, causing the application to crash or possibly leak sensitive data to the attacker. If the overwritten memory buffers contain function pointers or other control data, attackers can go as far as hijacking the program’s execution flow, resulting in the dreaded arbitrary code execution attack. In the case of **CVE-2023-5474** , arbitrary code execution was a possible outcome. If the attack involved a victim with escalated privileges in their system, it could allow the attacker to install new programs, view and change sensitive data, or even create entirely new user accounts.

#### Mitigating Vulnerabilities like CVE-2023-5474

Google Chrome has since patched this specific vulnerability with application updates. Regardless, the significance of **CVE-2023-5474** – and other vulnerabilities which follow a similar pattern – cannot be understated. No viruses or malware are directly involved in the process of exploiting this type of CVE, meaning traditional antivirus software won’t identify any threats in the file the attacker crafts to trigger the heap buffer overflow. Mitigating a threat like this requires:

  1. Regularly checking the affected application for updates (in this case, the Google Chrome web browser) and installing them.
  2. Verifying that all PDFs reaching sensitive locations (i.e., where a user might be able to open them) are thoroughly verified to ensure their contents rigorously conform to PDF formatting standards.



#### Scanning and Verifying Malicious PDFs with Cloudmersive

The [Cloudmersive Advanced Virus Scan API](https://cloudmersive.com/virus-api) combines virus and malware scanning with in-depth content verification to provide 360-degree protection against files entering an application (or network). Performing a dynamic threat scan on PDFs intended to exploit vulnerabilities like **CVE-2023-5474** can identify issues with the file that may have otherwise resulted in zero-day heap buffer overflow attacks in any one of the PDF rendering or parsing programs that our applications utilize in the background. For more information on how the Cloudmersive Advanced Virus Scan API can help protect your applications or network against invalid file threats, please do not hesitate to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[CVE Spotlight: Understanding CVE 2023-4863 Libwebp Heap Overflow ](/blog/CVE-Spotlight%3a-Understanding-CVE-2023-4863-Libwebp-Heap-Overflow-)  
4/1/2024 - Brian O'Neill  
  
  
![magnifying glass on lock graphic](https://docs.cloudmersive.com/api/image/blog-image/261ee3c7-da9a-4a6a-836a-675f62b18b6a) Just a few months ago, in September 2023, a critical (high-severity) [heap overflow](https://cloudmersive.com/article/What-is-a-Heap-Based-Buffer-Overflow-Vulnerability%3f) vulnerability was discovered in certain versions of Libwebp – a popular image codec library used in a variety of common applications, including web browsers like Google Chrome and Microsoft Edge. Troublingly, in the analysis of **CVE 2023-4863** , it was found that threat actors could exploit this particular heap overflow vulnerability without requiring any interaction from a user. Using a specially crafted malicious WebP file (and without authoring malware), a threat actor could force the decoding algorithm used in vulnerable versions of the Libwebp library to write data outside the scope of its usual memory allocation process. If an instance of software using a vulnerable Libwebp version were to process the malicious WebP file, it could lead to a Remote Code Execution or Denial of Service attack. Thankfully, shortly after this alarming vulnerability was identified, a new version of Libwebp was released to patch it. As ever, successfully avoiding attacks that exploit heap overflow flaws like **CVE 2023-4863** often boils down to religiously checking for relevant software updates and applying them as soon as they're available.

#### Learning from **CVE 2023-4863**

The sudden discovery of obviously severe vulnerabilities like **CVE 2023-4863** highlights an unfortunate truth in modern cybersecurity: motivated threat actors will tirelessly attempt to identify new, unseen vulnerabilities in the software libraries we passively trust, and threat researchers may not be able to discover and publish information about these vulnerabilities before devastating attacks occur. The incentive to find and exploit vulnerabilities like **CVE 2023-4863** is strong, after all. For most threat actors, exploiting unknown heap overflow vulnerabilities with specially crafted payloads is far simpler than authoring malware capable of bypassing the increasingly robust, sophisticated antivirus technologies potential victims can purchase on the modern cybersecurity market. Specially crafted files intended to exploit heap overflow vulnerabilities like the **CVE-2023-4863** example will, in all likelihood, bypass the antivirus technologies our applications rely on to ward off malicious file content, and the results of such a breach can be just as devastating as any virus or malware attack.

#### Cloudmersive Content Verification

Thankfully, one way we can mitigate attacks using specially crafted files to exploit heap overflow vulnerabilities is by incorporating content verification policies in our virus and malware file scans. In doing so, we can ensure files conform to rigid file formatting standards without needing to stay on top of image processing exploits quite as regularly. The [Cloudmersive Advanced Virus Scan API](https://cloudmersive.com/virus-api) combines in-depth content verification policies with powerful virus and malware detection capabilities to identify specially crafted file threats. For more information on Cloudmersive Advanced Virus Scan API content verification capabilities, please do not hesitate to [reach out to a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[API Preview: Cloudmersive Phishing API Coming Soon](/blog/API-Preview%3a-Cloudmersive-Phishing-API-Coming-Soon)  
3/15/2024 - Brian O'Neill  
  
  
We’re excited to announce that a brand-new product will soon be joining our portfolio of powerful cybersecurity APIs. The Cloudmersive Phishing API – currently in alpha – is designed to make identifying and blocking phishing URL threats a breeze. ![phish hook](https://docs.cloudmersive.com/api/image/blog-image/1f4f347e-f2a6-4482-843b-b5c3b798a34b)

#### Basic vs. Advanced Phishing API Capabilities

Two versions of the Phishing API will be available upon launch: a basic version, and an advanced version. These versions will differ in the scope of their phishing threat detection capabilities.

##### Basic Phishing API Lookups

The basic Phishing API will perform rigorous phishing URL lookups, referencing a continuously updated database of URLs, hosts, and domains to flag known phishing threats.

##### Advanced Phishing API Lookups

The advanced Phishing API will build upon the basic URL lookup service, bringing several unique phishing threat detection capabilities to the table. Some of these unique capabilities will be powered by Cloudmersive domain and URL validation APIs (available under the Cloudmersive Validate API umbrella). Advanced scans will check URLs for suspicious formatting abnormalities (e.g., Unicode characters disguised as Latin characters) and investigate if URLs are hidden redirection links. In the event redirection links are discovered, the advanced Phishing API will check the host and domain of the final URL redirect to fully diagnose the phishing threat. An option to allow or block specific domains will also be available in the advanced Phishing API’s request parameters.

#### Batch Phishing API Request Capabilities

Both the basic and advanced versions of the Phishing API will have batch request API iterations available to facilitate processing more than one suspicious URL per request. Batch API calls will compartmentalize each input URL with its own nested threat assessment object in the API response.

#### Learn More about the Cloudmersive Phishing API

Interested in learning more about the Cloudmersive Phishing API? Please do not hesitate to [contact a member of our sales team](https://portal.cloudmersive.com/contact-sales) with any questions you might have.  
  
  
  
[How to Virus Scan Azure Blob Storage Uploads in Power Automate](/blog/How-to-Virus-Scan-Azure-Blob-Storage-Uploads-in-Power-Automate)  
3/6/2024 - Brian O'Neill  
  
  
In Power Automate, we have the convenient option to call the Cloudmersive Virus Scan API connector in a simple cloud flow. Below, we’ll set up an example automated flow in Power Automate that scans Azure Blob Storage uploads to a specific blob container in our Azure Storage account. _Please note – a premium Power Automate subscription is required to utilize Cloudmersive connectors_. Let’s start off on the Power Automate home page. From here, let’s select **Create** on the lefthand side of the page to bring up our flow type options. ![1 - select create](https://docs.cloudmersive.com/api/image/blog-image/7df88853-50ba-4129-ad51-ea45a2d03806) We’re going to create a flow that automatically triggers when a new file is added to our Azure Blob Storage Container, so let’s select the **automated cloud flow** option. In the **build an automated cloud flow window** , let’s give our flow a relevant name (i.e., Virus Scan Azure Blob Stroage Uploads), and then let’s type “blob” into the **Chose your flow’s trigger** search bar. ![2 - searching for blob trigger](https://docs.cloudmersive.com/api/image/blog-image/9a792c59-990d-40ca-97cc-906bc7f887f3) This brings up the **When a blob is added or modified (properties only) (V2)** trigger, which is the only trigger action available in the **Azure Blob Storage** connector. Let’s select this option and click **Create** to start designing our flow. On the flow diagram page, we’ll first need to configure our Azure Blob Storage connection before we can configure our request parameters. ![3 - configure azure blob connection with access key](https://docs.cloudmersive.com/api/image/blog-image/dbc8f631-0493-4b97-80dc-ebc4e2bad501) In this example, we’ve selected “Access Key” from the **Authentication type** dropdown (our other options include “Service principal authentication” and “Microsoft Entra ID Integrated”). This way, we’ll just need to provide our **Azure Storage account name** or **blob endpoint** along with our **Azure Storage Account Access key**. We can find our **Access key** information in the **Security + networking** section of our Azure Blob Storage account page. Once we’ve created our connection, we can open our trigger parameters tab and select the **Storage Account Name** or **Blob Endpoint** we’re targeting in our scan. Below that, we can select our target **Container** by following our Azure Blob Storage folder path. ![4 - configure trigger parameters](https://docs.cloudmersive.com/api/image/blog-image/09475aac-cc4b-49dd-9a5a-c57d861cfcc7) If we click **Show all** next to the **Advanced parameters** dropdown, we’ll have the option to specify the number of blobs we want to return in our trigger. This option (along with the dropdown menu below it which provides the option to specify trigger intervals) is primarily advantageous for storage containers which deal with a large number of files, so we’ll be leaving it blank in this example workflow. With our trigger step successfully configured, our flow is now going to retrieve properties from new blobs as they enter our target Azure Blob Storage container. The Cloudmersive Virus Scan Connector requires file content to perform a scan, so we’ll need our next action to retrieve blob contents using blob properties. Let’s click on the **Add an action** button below our flow trigger and type “Azure Blob Storage” into the search bar. ![5 - find azure blob storage connector](https://docs.cloudmersive.com/api/image/blog-image/b518d21b-97e5-4063-98d0-45f6756049b1) This brings up the **Azure Blob Storage** connector, and we can click **See more** to enumerate a list of connector actions. ![6 - azure blob storage actions list](https://docs.cloudmersive.com/api/image/blog-image/cf435d0b-e849-40d3-b8ae-bb3fda5f70f5) We’re looking for the action called **Get blob content using path (V2)** , which we can find towards the middle-bottom of this list. Let’s click on it and begin configuring the request parameters. ![7 - configure get file content parameters](https://docs.cloudmersive.com/api/image/blog-image/ee9764fd-709c-4e25-98cc-b81d25fe0a64) Just like we did in our trigger step, we’ll first need to select our **Storage Account Name** or **Blob Endpoint** from the initial dropdown. Below that, in the **Blob Path** parameter, let’s select dynamic content from our trigger step labeled **List of Files Path** , and then let’s leave the **Infer Content Type** dropdown on “Yes”. ![8 - displaying for each control](https://docs.cloudmersive.com/api/image/blog-image/4a81ba1b-66e5-4ef3-91a4-7f4fecc5cc7c) We’ll notice that selecting the **List of Files Path** content wrapped our **Get blob content using path (V2)** action in a **For each** control to account for the possibility that multiple files were uploaded to our target Azure Storage Container. Within the **For each** control, let’s click on the **Add an action** button and search for the **Cloudmersive Virus Scan** connector. ![9 - search for cloudmersive virus scan connector](https://docs.cloudmersive.com/api/image/blog-image/ab51947d-04f2-47bc-b183-350b96ab3441) From the actions we see available under the **Cloudmersive Virus Scan** connector, we can click on the **Scan a file for viruses** action. If we haven’t used this connector before, we’ll need to name our connection and enter our API key in the ensuing connector configuration step. Once we do so, we’ll be able to configure our Virus Scan request parameters. ![10 - configure cloudmersive virus scan a file action](https://docs.cloudmersive.com/api/image/blog-image/5a859d2a-ef83-47de-93df-71d1b3de835f) Within the **Input File** parameter, let’s select dynamic content labeled **File Content** from our **Get blob content using path (V2)** step. When we save and test this flow (by uploading a new file to our Azure Blob Storage container), the content retrieved from each new blob item path will now undergo a rigorous virus & malware scan. ![11 - successful test](https://docs.cloudmersive.com/api/image/blog-image/b8e6ca80-f3be-4da4-aa41-a372fda8fd0b) Because we performed a basic scan with the Cloudmersive Virus Scan API, our flow output will contain a “CleanResult” response which can either be **True** or **False**. ![12 - cleanresult true or false](https://docs.cloudmersive.com/api/image/blog-image/04be6862-62bf-4785-a02c-b37a7604f35b) This example flow was tested with a clean Excel XLSX file upload, resulting in a “CleanResult”: **true** response. Depending on the security limitations in our environment, we can trigger “CleanResult”: **false** responses using EICAR files from the EICAR website. We can use the “CleanResult” Boolean response to take various actions as a next step in our flow. We could, for example, ask Azure to delete the file immediately in the event of a “CleanResult”: **false** response, and to do nothing in the event of a **true** response. To create this condition, we would first need to add a new action within our **For each** control and search for the **Condition** control. ![13 - search for condition control](https://docs.cloudmersive.com/api/image/blog-image/12a14066-297d-4566-b74b-35aaa35afa30) When configuring our condition, we could set dynamic content labeled **body/CleanResult** from our **Scan a file for viruses** step equal to “false”. ![14 - configuring condition control](https://docs.cloudmersive.com/api/image/blog-image/c9cd8365-61d2-482d-acf6-3906196a4012) Then, within the **True** outcome of our condition (meaning a “CleanResult”: **false** response did occur), we could add an action to delete the blob from our Azure Blob Storage container. This action can be found in the **Azure Blob Storage** connector actions list, and it can be configured by once again selecting our **Storage Account Name** or **Blob Endpoint** from a dropdown before filling the **Blob** parameter with dynamic content labeled **List of Files Path** from our trigger step. ![15 - final condition with delete blob action](https://docs.cloudmersive.com/api/image/blog-image/7036087c-8053-4550-88b6-34714394ec09) Now we have a simple, automated flow running in the background that scans and deletes virus- or malware-infected blobs from an Azure Blob Storage container. For more information on the Cloudmersive Virus Scan API, please do not hesitate to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Announcing New Cloudmersive Virus Scanning Reverse Proxy Policies](/blog/Announcing-New-Cloudmersive-Virus-Scanning-Reverse-Proxy-Policies)  
2/14/2024 - Brian O'Neill  
  
  
![Generic blue hexagons](https://docs.cloudmersive.com/api/image/blog-image/4545cad0-388d-489c-8841-1b18e6dda774) We’re pleased to announce that new policies have been added to the Cloudmersive Virus Scanning Reverse Proxy. A new security policy was added to block specific paths, and a new transform policy was included to response redirect on request path. The Cloudmersive Virus Scanning Reverse Proxy Server provides unparalleled 360-degree protection at the network edge, scanning incoming files for viruses, malware, and non-malware content threats. For more information on the Cloudmersive Reverse Proxy or other Cloudmersive Virus Scan API deployments, please do not hesitate to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Announcing Enhanced Virus Scan API Password Detection for XLS and DOC Files](/blog/Announcing-Enhanced-Virus-Scan-API-Password-Detection-for-XLS-and-DOC-Files)  
2/7/2024 - Brian O'Neill  
  
  
![Generic blue tech binary](https://docs.cloudmersive.com/api/image/blog-image/6ca37fd7-64eb-4804-ab25-4d1516418b78) We’re pleased to announce that enhanced password detection capabilities have been added to the Cloudmersive Advanced Virus Scan API in a recent update. Password-protected files are commonly utilized to obfuscate malicious code from regular antivirus software. Without proper content verification policies in place, threat actors can potentially slip password protected files into sensitive storage locations and supply passwords to trigger arbitrary code execution at a later date. Enhanced password detection capabilities now ensure legacy Excel and Word file formats can be flagged alongside the dozens of modern Office, image, and PDF file formats already supported by the Cloudmersive Advanced Virus Scan API. For more information on content verification with the Cloudmersive Advanced Virus Scan API, please do not hesitate to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Vulnerability Spotlight: Understanding CVE-2023-21608 PDF Reader Flaw](/blog/Vulnerability-Spotlight%3a-Understanding-CVE-2023-21608-PDF-Reader-Flaw)  
2/2/2024 - Brian O'Neill  
  
  
In the final quarter of 2023, a use-after-free vulnerability was identified in (and before) several versions of Adobe Acrobat Reader. Without involving viruses or malware whatsoever, successful exploitation of this type of vulnerability could have very severe consequences. ![Magnifying glass on red item](https://docs.cloudmersive.com/api/image/blog-image/78d32f3a-95f2-4f94-8a06-d2af99a02b83) This vulnerability could, for one example, lead to arbitrary code execution in the context of the victim (current application user loading a PDF in Reader). If that victim had escalated privileges within the target application, the attacker could end up with access to large volumes of sensitive data. Let’s take a closer look at what a “use-after-free” vulnerability really means, and let's understand how we can mitigate threatening content that might be used to exploit zero-day use-after-free vulnerabilities.

#### What is a use-after-free vulnerability?

To manage memory efficiently, computer programs usually release (deallocate) memory which was first dynamically allocated during the program’s runtime. Many relevant programming languages in this context use a function (such as **free()** or **delete**) to release memory, and once that release occurs, the computer program effectively “assumes” it won’t be accessing (or “pointing” to) that memory block again. When there’s a use-after-free vulnerability in an application, it means there’s a flaw making it possible to force the program to continue “pointing” to a memory block that was already deallocated from use. Once the program starts trying to access deallocated memory, unpredictable things can happen. Attackers can take advantage of this flaw to execute arbitrary code and possibly change the program’s behavior entirely.

#### How could a threat actor exploit a use-after-free vulnerability like CVE-2023-21608?

To exploit a use-after-free vulnerability in a PDF reader application, a threat actor would start by crafting a malicious PDF file. This file would likely contain a specific set of characteristics (for example: special objects, structures, or scripts/instructions) designed to exploit insecure memory deallocation functions. The attacker would then disguise and possibly obfuscate this file’s content so it appeared as safe & innocuous as possible (likely in addition to leveraging social engineering techniques) before sharing it with a potential victim using a vulnerable version of the PDF reader application. Once the victim opened the malicious PDF in the vulnerable PDF reader application, the PDF’s special elements could force the application to “point” to deallocated memory. The attacker could then take control of this freed memory execute arbitrary code within the vulnerable program. If the attacker chose their victim wisely, they could have escalated privileges within the vulnerable application and potentially take complete control of it. If this all sounds a little too easy, we’re on the right track: once a sophisticated threat actor is aware of a use-after-free vulnerability in a particular application, they’ve already done the hardest work. Disguising and/or obfuscating malicious files & employing effective social engineering techniques to exploit targeted victims are often trivial tasks for experienced attackers, largely because no virus or malware signatures are involved. Traditional antivirus software very likely won’t find anything wrong with the malicious PDF, and the target user might assume the file was safe if it appeared to pass a standard antivirus scan.

#### How can we mitigate the exploitation of use-after-free vulnerabilities?

First and foremost, it’s critical that we update all our applications regularly. Security teams, researchers and bug bounty hunters are constantly at work trying to identify critical vulnerabilities in popular applications, and once those vulnerabilities are identified, new software versions are hastily released with relevant fixes. To avoid becoming the subject of a zero-day use-after-free vulnerability exploit, however, we need to focus on rigorously verifying the content processed by our sensitive applications. Use-after-free vulnerability exploits might be driven by insecure coding practices, but they only result in outcomes like arbitrary code execution when malicious content isn’t properly validated before passing through program functions. Unintentional (non-malicious) use-after-free exploits can occur because of improperly validated content too, potentially leading to other unwanted outcomes such as Denial-of-Service.

#### Verifying content with the Cloudmersive Virus Scan API

The advanced version of the Cloudmersive Virus Scan API performs in-depth content verification checks on files & identifies the types of content threat actors use to exploit vulnerabilities like use-after-free. It can be configured to disallow content containing scripts, executables, invalid file content (relative to the file extension), JavaScript, HTML, and more. In the context of protecting a potentially vulnerable PDF reader application, API request parameters could be customized to flag PDFs containing any content that did not fully conform to PDF standards. This means scripts, executables, and HTML/JavaScript objects would be called out before the file could reach the target application. For more information on the Cloudmersive Virus Scan API, please feel free to [reach out to a member of our team](https://cloudmersive.com/).  
  
  
  
[Announcing STEP File Content Verification Support for the Cloudmersive Virus API](/blog/Announcing-STEP-File-Content-Verification-Support-for-the-Cloudmersive-Virus-API)  
1/25/2024 - Brian O'Neill  
  
  
![lock with arrows pointing both ways](https://docs.cloudmersive.com/api/image/blog-image/fa6d8f64-fab2-4a7a-b5f0-51700acb6f40) We’re pleased to announce our newest Cloudmersive Virus API (v8-5-4) update release has incorporated support for STEP (Standard for the Exchange of Product model data) files. STEP files (i.e., those with .STEP or .STP extensions) scanned through the Virus API will now undergo content verification checks to ensure 1) these files' contents rigorously conform to STEP file standards, and 2) these files do not host various hidden threats including scripts, executables, external links, and other potentially malicious content. The Cloudmersive Virus API offers 360-degree content protection. That starts with virus and malware scanning and extends to full in-depth content verification for executables, invalid files, scripts, password protected files, macros, XML external entities, insecure deserialization, HTML contents, unsafe archives, OLE embedded objects, and PDFs with JavaScript/HTML enabled. For more information on the Cloudmersive Virus API, please do not hesitate to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Mitigating Invalid Image File Threats with the Cloudmersive Advanced Virus Scan API](/blog/Mitigating-Invalid-Image-File-Threats-with-the-Cloudmersive-Advanced-Virus-Scan-API)  
1/10/2024 - Brian O'Neill  
  
  
![man typing in suit with graphics](https://docs.cloudmersive.com/api/image/blog-image/0b2a2099-a05e-4394-8537-365f73662a15) With an estimated 500,000+ new malware signatures developed each day, implementing effective cyber-security policies is an extremely difficult task – but our list of threats doesn’t end there. Malware is, unfortunately, only one of many weapons found in a threat actor’s cyber-attack arsenal.

#### Understanding the Threat of Invalid Image Files

Image format manipulation is one example of a critical non-malware attack vector which most antivirus solutions aren't configured to detect. By crafting malicious image files with special formatting, threat actors can exploit zero-day vulnerabilities in the libraries and tools we rely on to process and buffer image uploads & downloads. These vulnerabilities can sometimes crash our applications or pave the way for arbitrary code execution.

#### Examples of Image Formatting Vulnerabilities

There have been many examples of critical image processing vulnerabilities in recent years, and we'll highlight two significant examples here. One example we can look at is a heap-based overflow vulnerability (CVE-2023-4863) that was recently identified within **libwebp** , a WebP codec library commonly used in web browser applications. By crafting a malicious WebP file, a threat actor could ignore heap buffer boundaries in libwebp and cause the target application to crash, resulting in a denial-of-service attack. We can also look at a somewhat similar vulnerability (CVE-2022-3570) found one year prior in **tiffcrop** , a utility within the **libtiff** library used for selecting, copying, and cropping TIFF image files. Using a specially crafted TIFF file, an attacker could exploit this vulnerability to enable out-of-bounds access in the target application, allowing them to steal sensitive information from their attack victim.

#### Detecting Invalid Image File Threats

Unlike files infected with viruses or malware, invalid image files aren’t going to get flagged by our regular antivirus software. Rather, threats of this nature can be mitigated more effectively with rigorous content validation policies. If we can determine that image files are flawed before they reach a potentially vulnerable image processing technology, we can eliminate the threat of those files ignoring heap buffer boundaries, enabling out-of-bounds access, or triggering other contextually exploitative scenarios. The need for stringent content validation adds a challenging parameter to our existing content security architecture. Thankfully, however, we don’t necessarily need to think of antivirus scanning and content validation separately. If we can perform antivirus scans AND validate file contents simultaneously, we can significantly improve our threat profile while reducing the cost (in time and resources) of managing multiple separate cyber-security solutions.

#### Content Verification & Validation with the Cloudmersive Advanced Virus Scan API

The Cloudmersive Advanced Virus Scan API combines a cutting-edge malware detection sandbox with custom content validation parameters, resulting in unique 360-degree protection for our image uploads and downloads (this service also supports dozens of additional document types, including all Office file formats and PDFs). Invalid image files can be flagged through the **allowInvalidFiles** parameter, and we can restrict myriad unwanted image formats altogether by entering a comma-separated whitelist of acceptable format types (e.g., “.png,.webp,.jpg”) into the **restrictFileTypes** parameter. This API can be deployed as a no-code solution in a variety of strategic locations, including at the network edge (i.e., network proxies & ICAP servers) or adjacent to sensitive cloud storage buckets, and it can be accessed directly as a low-code solution through API calls to the cloud. This dynamic threat scanning service can significantly improve our threat profile. For more information on the Cloudmersive Virus Scan API, please feel free to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Announcing Azure Blob Storage Accounts for Cloudmersive Reverse Proxy Servers](/blog/Announcing-Azure-Blob-Storage-Accounts-for-Cloudmersive-Reverse-Proxy-Servers)  
12/20/2023 - Brian O'Neill  
  
  
With the recent release of Cloudmersive Reverse Proxy v3.3.8, we’re excited to announce the addition of a brand-new feature! Azure Blob Storage accounts can now function as the origin accounts for Cloudmersive Reverse Proxy Servers. This will allow customers to serve their Azure Blob Storage containers through their Cloudmersive Reverse Proxy server, with access controlled via SAS token. That’s not all, either. With this release, we’ve included several new Cloudmersive Reverse Proxy Server policies and one update to an existing policy. These including the following:

  1. Customers can now block specific HTTP request methods through an **HTTP Request Method Policy**.
  2. Customers can now block specific HTTP response codes returning from the origin server using an **HTTP Response Code Policy**.
  3. The existing **URL Replace** policy has been updated to include **Match Mode** setting, which allows customers to choose between _substring match_ and _exact match_.

For more information on new Cloudmersive Reverse Proxy features & updates, please do not hesitate to [reach out to a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Announcing Cloudmersive Reverse Proxy for Windows Server 2022](/blog/Announcing-Cloudmersive-Reverse-Proxy-for-Windows-Server-2022)  
12/13/2023 - Brian O'Neill  
  
  
![blue server room angled right](https://docs.cloudmersive.com/api/image/blog-image/d6127185-3cfc-4d5c-81eb-252914d29947) We’re pleased to announce that the Cloudmersive Reverse Proxy server is now available for Windows Server 2022. This latest addition joins Windows Server 2016, Debian 10 Linux, and Kubernetes on our list of compatible servers. The Cloudmersive Reverse Proxy Server (also known as Cloudmersive Shield) is a no-code solution that can be deployed in front of your applications to block malicious files at the network edge. It allows you to protect any web application or API with zero code changes. Cloudmersive Shield will automatically detect file uploads and send them to the Cloudmersive Virus Scanning API for virus scanning and content verification. When files are clean, they will continue to the client application, and when files are infected, a customized response will be displayed to the user. Shield additionally features the ability to scan binary data in JSON and XML requests for threats. For more information regarding the Cloudmersive Reverse Proxy Server, please do not hesitate to [reach out to a member of our team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Announcing India: The Newest Cloudmersive Data Center Region](/blog/Announcing-India%3a-The-Newest-Cloudmersive-Data-Center-Region)  
11/27/2023 - Brian O'Neill  
  
  
We are pleased to announce the launch of a brand-new Cloudmersive data center region in **Mumbai, India**. This new region joins our existing data centers in **Singapore** and **Sydney, Australia** to comprehensively serve clients located throughout South Asia and Oceania. Our [global data center deployment regions](https://cloudmersive.com/regions) now include all locations depicted in the graphic below: ![Data Center Regions](https://docs.cloudmersive.com/api/image/blog-image/32ac1d4b-8bfc-4c49-a290-b31197f8167d)

#### North America

  * Hillsboro, Oregon, US
  * Beauharnois, Quebec, Canada
  * Vint Hill, Virginia, US



#### Europe

  * London, England, UK
  * Gravelines, Hauts-de-France, France
  * Frankfurt, Hesse, Germany
  * Warsaw, Masovia, Poland



#### Asia

  * Mumbai, India
  * Singapore



#### Australia

  * Sydney, New South Wales, Australia

For more information on data center regions and Cloudmersive deployment options, please do not hesitate to [contact a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Spotlight: 360-Degree Content Protection](/blog/Spotlight%3a-360-Degree-Content-Protection)  
11/6/2023 - Brian O'Neill  
  
  
![content graphic](https://docs.cloudmersive.com/api/image/blog-image/d8ddbdef-b5fd-4013-ac5f-af88eb6bbe43) Every day, threat actors across the globe come up with an astonishing number of new malware strains – a volume estimated in the hundreds of thousands – and they also come up with increasingly creative & effective ways to hide that malware within common file types that we regularly use for legitimate purposes. If our scanning service is bypassed even once by a cleverly crafted password-protected file or an encrypted zip archive, the results can be disastrous, leaving us struggling to pick of the pieces from a devastating cyberattack. Thankfully, however, when we combine shrewd content restrictions with our regular malware detection processes, we can establish a new level of control over the content that enters and leaves our system. Instead of relying exclusively on the analysis of threat signatures and suspicious file behaviors, we can categorically block file types that we know we don’t want to deal with in any capacity. We can simply remove password-protected files, executable files, scripts, and other vulnerable, commonly exploited entities from our workflow, and we can even go as far as to whitelist the files we _do_ want to accept and block everything else entirely. We can accomplish all of this by rigorously verifying file contents against the file extension they present – a process which simultaneously blocks targeted file types and uncovers invalid file types with spoofed extensions. The advanced iteration of the Cloudmersive Virus Scan API provides powerful 360-degree content protection, combining cutting-edge malware scanning and in-depth content verification into one simple, high-speed scanning process with unprecedented customization options. Whether we’re using Cloudmersive Storage Protect to scan our AWS, Azure, SharePoint & GCP storage locations, or using Cloudmersive Shield to safeguard our forward and reverse network proxies, we can depend on powerful content protection without writing a single line of code.  
  
  
  
[Understanding Cloudmersive Private Cloud Deployment](/blog/Understanding-Cloudmersive-Private-Cloud-Deployment)  
10/25/2023 - Brian O'Neill  
  
  
Flexibility is a key component of any Cloudmersive product deployment. Choosing the Cloudmersive Private Cloud Deployment option provides customers with the most control over their own data & security model. In a Private Cloud Deployment, customers install their Cloudmersive product either on-premises or through their choice of cloud provider (e.g., AWS, Azure, GCP), and they are ultimately responsible for managing and updating their Cloudmersive service. In this deployment model, the Cloudmersive team will provide walk-through guidance for the customer as needed during installation and updates. Much like the [Managed Instance Deployment](https://cloudmersive.com/article) option, the Private Cloud Deployment option is fully customizable with no pre-set limits, and it’s a highly scalable solution licensed per core per year. For more information on Cloudmersive Private Cloud Deployment, please do not hesitate to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Announcing JavaScript & HTML Policies for PDF Content Verification](/blog/Announcing-JavaScript-%26-HTML-Policies-for-PDF-Content-Verification)  
10/16/2023 - Brian O'Neill  
  
  
We are pleased to announce that the list of content verification policies available to Storage Protect & Shield customers has been expanded with the release of version 8-2-8 of the Cloudmersive Virus Scan API. Through the **permitJavascriptAndHtmlinPDFs** request parameter, customers can now set restrictions against PDFs containing JavaScript or HTML code. When this policy is active, files with .PDF extensions that contain HTML or JavaScript within their file structure will fail their content verification check and trigger a **CleanResult: False** response. Malicious PDF files containing JavaScript and HTML can be used to bypass virus & malware scanning policies, and this can leave internal or external network users at risk of inadvertently accessing malicious content and initiating a cyberattack. The option to categorically restrict PDF files containing JavaScript and HTML is another convenient custom-content threat prevention solution for customers with multi-pronged content security concerns.

#### More on the Cloudmersive Virus Scan API

The Cloudmersive Virus Scan API offers customers a uniquely comprehensive file scanning service, combining dynamic malware threat detection capabilities with customizable in-depth content verification policies. In addition to gaining state-of-the-art, zero-day file threat detection capabilities with their baseline Cloudmersive Virus Scan API product, customers utilizing the Advanced Virus Scan API service (through any deployment type) can easily block executables, invalid files, scripts, password-protected files, and more using simple custom request parameters. They can additionally whitelist acceptable file formats by specifying a comma-separated plain text list of file extensions. For more information about the Cloudmersive Virus Scan API, please feel free to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Server-Side Request Forgery: Back in the OWASP Top 10](/blog/Server-Side-Request-Forgery%3a-Back-in-the-OWASP-Top-10)  
10/5/2023 - Brian O'Neill  
  
  
![blue dotted scape](https://docs.cloudmersive.com/api/image/blog-image/6fa0206a-7952-42e2-8d7d-9d92a399d6fa) The most recent edition of OWASP’s top-10 security vulnerabilities was published in July of this year (2023), and a familiar threat was once more included in that ranking. Coming in at 7th this year – three slots higher than the 10th place slot it occupied in the [2021 OWASP ranking](https://cloudmersive.com/blog/OWASP-Top-10-2023%3a-A-Brief-Overview-of-New-and-Returning-Web-Application-Security-Risks) – was Server-Side Request Forgery ([SSRF](https://cloudmersive.com/article/What-is-a-SSRF-\(Server-Side-Request-Forgery\)-Attack%3f)), a well-known vulnerability that enables attackers to control where server-side applications make requests. This type of attack is often used to leak important/sensitive data from an organization by exploiting trust relationships between internal servers. There are a few key steps we can take to mitigate SSRF vulnerabilities, such as altering the way we represent the 127.0.0.1 IP or utilizing URL encoding techniques to obscure our blocked strings. In addition, we can take advantage of the **Cloudmersive Security API** for advanced network threat detection. We can use the SSRF Detection iteration of the Security API to check if an URL is at risk of being an SSRF attack, and we can input blocked domains along with our URL string request. The API will return a **CleanURL** Boolean response, along with a **Threat Level** description string. Like all Cloudmersive APIs, the Security API is a turnkey, low-code solution designed to be easy for developers to use. For more information about deploying the Cloudmersive Security API, please do not hesitate to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Set up Cloudmersive Storage Protect for AWS S3 and Azure Blob in Minutes](/blog/Set-up-Cloudmersive-Storage-Protect-for-AWS-S3-and-Azure-Blob-in-Minutes)  
9/20/2023 - Brian O'Neill  
  
  
![cloud security ](https://docs.cloudmersive.com/api/image/blog-image/3fe0ba02-99ad-4b9f-a293-1c5e1c689572) Storing client-side file uploads in the cloud is becoming an increasingly risky business. As more and more companies purchase cloud storage subscriptions from Amazon, Microsoft, and other major industry players, threat actors divert more and more attention toward exploiting these storage locations with insecure file uploads & custom threats designed to bypass peripheral high-level virus scanning policies. In-storage virus scanning & content verification can be the difference between the success and failure of a latent cyber-attack. By deploying **Cloudmersive Storage Protect** alongside our buckets and blobs, we can scrutinize each individual file upload in-storage, detecting virus and malware signatures & simultaneously identifying invalid files, scripts, password-protected files, archives, and a variety of additional hidden content threats. We can even set custom restrictions against unwanted file types to dramatically increase our threat profile – all without writing a line of code. If you’re considering in-storage protection for AWS S3 or Azure Blob storage, check out the below videos to see how you can get **Cloudmersive Storage Protect** up and running in minutes. [How to Scan AWS S3 Files for Viruses](https://cloudmersive.com/videos/How-to-Scan-AWS-S3-Files-for-Viruses) [How to Scan an Azure Blob for Viruses](https://cloudmersive.com/videos/How-to-Scan-an-Azure-Blob-for-Viruses)  
  
  
  
[OWASP Top 10 2023: A Brief Overview of New and Returning Web Application Security Risks](/blog/OWASP-Top-10-2023%3a-A-Brief-Overview-of-New-and-Returning-Web-Application-Security-Risks)  
9/12/2023 - Brian O'Neill  
  
  
Since 2003, OWASP (the Open Worldwide Application Security Project) has, every few years, released a ranking of its top 10 most relevant web application security risks. These rankings are informed entirely by contemporary research and feedback accumulated from the non-profit’s global “open community” of dedicated cybersecurity professionals, and they tend to impact the way organizations around the world analyze and plan out their future web application security practices. The most recent OWASP Top 10 API Security Risks ranking, published in early July 2023, features several changes from the previous ranking published in September 2021. Included on this list are several returning threats - some of which have been renamed - along with five new additions. The below chart depicts the OWASP 2021 & 2023 rankings side by side with color-coded cells for new and returning threat inclusions: ![OWASP top 10 2021 and 2023 comparison](https://docs.cloudmersive.com/api/image/blog-image/7f104797-ec80-46f1-9cc6-509a4097d085) In the above chart, the five threats removed from the 2021 list are highlighted in **red** , while the new additions to the 2023 list are highlighted in **green**. Of the returning inclusions, the three renamed threats are highlighted in **beige** on both lists, while the two unchanged inclusions are highlighted in **light blue** on both lists. **Broken Access Control** is now referred to as **Broken Object Level Authorization** , while **Identification and Authentication Failures** are now referred to as **Broken Authentication** and **Vulnerable and Outdated Components** are renamed as **Improper Inventory Management**. For more information regarding Cloudmersive API Security practices & solutions for mitigating common web application security risks, please do not hesitate to [contact a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Make the Most of your PDF Documents in Power Automate](/blog/Make-the-Most-of-your-PDF-Documents-in-Power-Automate)  
9/4/2023 - Brian O'Neill  
  
  
If there's one thing every client device in the world has in common, it's the ability to open and view PDF documents on a browser. That interoperability is present by design, and it makes PDF one of the most valuable content sharing formats in the world. In Power Automate, we can build efficient workflows around converting, editing, and optimizing PDFs, saving us even more time in our day-to-day file processing tasks. With Cloudmersive API Connectors at the center of our flow designs, we can elevate these processes to an even higher level. ![PDF connector actions](https://docs.cloudmersive.com/api/image/blog-image/b85d9000-6fbd-4c9a-a420-a4eae8d8f3bf) ![Doc Conversion PDF](https://docs.cloudmersive.com/api/image/blog-image/e8ac7dba-8f68-424f-a8fa-524053cccd68) Below, we've compiled a few documentation videos centered around our PDF connector actions in Power Automate to help you get up and running in minutes.

#### Convert To PDF

[Convert PowerPoint Presentations to PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/Convert-PowerPoint-Presentations-to-PDF-in-Power-Automate-and-Logic-Apps) [Convert Word Documents to PDF in Power Automate](https://cloudmersive.com/videos/Convert-Word-Documents-to-PDF-in-Power-Automate) [How to Convert a Single Excel Worksheet to PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-a-Single-Excel-Worksheet-to-PDF-in-Power-Automate-and-Logic-Apps) [How to Convert an Excel Spreadsheet to PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-an-Excel-Spreadsheet-to-PDF-in-Power-Automate-and-Logic-Apps) [Convert a PNG Image Array to PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/Convert-a-PNG-Image-Array-to-PDF-in-Power-Automate-and-Logic-Apps) [How to Convert a URL to a PDF using Power Automate](https://cloudmersive.com/videos/How-to-Convert-a-URL-to-a-PDF-using-Power-Automate) [How to Convert HTML to PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-HTML-to-PDF-in-Power-Automate-and-Logic-Apps)

#### Convert From PDF

[How to Convert PDF to PPTX in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-PDF-to-PPTX-in-Power-Automate-and-Logic-Apps) [How to Convert PDF to DOCX in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-PDF-to-DOCX-in-Power-Automate-and-Logic-Apps) [Convert a PDF Document to Text in Power Automate and Logic Apps](https://cloudmersive.com/videos/Convert-a-PDF-Document-to-Text-in-Power-Automate-and-Logic-Apps) [How to Rasterize a Text-Based PDF in Power Automate](https://cloudmersive.com/videos/How-to-Rasterize-a-Text-Based-PDF-in-Power-Automate) [How to Convert a Rasterized PDF to a DOCX File in Power Automate](https://cloudmersive.com/videos/How-to-Convert-a-Rasterized-PDF-to-a-DOCX-File-in-Power-Automate) [How to Convert a PDF File to a Single-Page PNG File in Power Automate](https://cloudmersive.com/videos/How-to-Convert-a-PDF-File-to-a-Single-Page-PNG-File-in-Power-Automate) [How to Get Text in a PDF Document by Page](https://cloudmersive.com/videos/How-to-Get-Text-in-a-PDF-Document-by-Page)

#### Edit PDF

[Merge PDF Documents in Power Automate and Logic Apps](https://cloudmersive.com/videos/Merge-PDF-Documents-in-Power-Automate-and-Logic-Apps) [How to Add a Text Watermark to a PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Add-a-Text-Watermark-to-a-PDF-in-Power-Automate-and-Logic-Apps) [How to Rotate All Pages in a PDF File in Power Automate](https://cloudmersive.com/videos/How-to-Rotate-All-Pages-in-a-PDF-File-in-Power-Automate) [How to Remove, Delete Pages from a PDF in Power Automate](https://cloudmersive.com/videos/How-to-Remove%2c-Delete-Pages-from-a-PDF-Document-in-Power-Automate) [How to Insert Pages from One PDF into Another PDF](https://cloudmersive.com/videos/How-to-Insert-Pages-from-One-PDF-into-Another-PDF) [How to Remove PDF Annotations, Comments in Power Automate](https://cloudmersive.com/videos/How-to-Remove-PDF-Annotations%2c-Comments-in-Power-Automate) [How to Get PDF Document Metadata in Power Automate](https://cloudmersive.com/videos/How-to-Get-PDF-Document-Metadata-in-Power-Automate) [How to Split PDF Documents into Separate Files (One per Page) in Power Automate](https://cloudmersive.com/videos/How-to-Split-PDF-Documents-into-Separate-Files-\(One-per-Page\)-in-Power-Automate) [How to Create PDF Annotations, Comments in Power Automate](https://cloudmersive.com/videos/How-to-Create-PDF-Annotations%2c-Comments-in-Power-Automate) [How to Remove a Specific PDF Annotation or Comment in Power Automate](https://cloudmersive.com/videos/How-to-Remove-a-Specific-PDF-Annotation-or-Comment-in-Power-Automate)

#### Validate & Get PDF Metadata

[How to Validate a PDF Document in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Validate-a-PDF-Document-in-Power-Automate-and-Logic-Apps) [How to Get PDF Metadata in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Get-PDF-Metadata-in-Power-Automate-and-Logic-Apps)

#### Encrypt and Decrypt PDF

[How to Decrypt a PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Decrypt-a-PDF-in-Power-Automate-and-Logic-Apps) [How to Encrypt, Password Protect, and Set Permissions for a PDF in Power Automate](https://cloudmersive.com/videos/How-to-Encrypt%2c-Password-Protect%2c-and-Set-Permissions-for-a-PDF-in-Power-Automate) For more information on Cloudmersive PDF APIs in Power Automate, please do not hesitate to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Convert, Split & Merge Your PowerPoint PPTX Presentations in Power Automate](/blog/Convert%2c-Split-%26-Merge-Your-PowerPoint-PPTX-Presentations-in-Power-Automate)  
8/31/2023 - Brian O'Neill  
  
  
![Presentation ](https://docs.cloudmersive.com/api/image/blog-image/431a6883-61e2-4811-b87d-b293f5c7bc9c) There’s a lot we can do to impact the design and export of PPTX presentations without ever opening a file in the PowerPoint application. With the help of Cloudmersive Document Conversion Connectors in Power Automate, we can automate a variety of vital PowerPoint actions and drastically simplify our file processing & file administration workflows. Don’t hesitate to check out a few of our demonstration videos on the subject – they’ll help you get up and running in minutes. ![Doc Conversion PPTX](https://docs.cloudmersive.com/api/image/blog-image/c217f144-db58-465e-b1d3-f1655f842ffd) [How to Convert a PowerPoint PPTX Document to Plain Text in Power Automate](https://cloudmersive.com/videos/How-to-Convert-a-PowerPoint-PPTX-Document-to-Plain-Text-in-Power-Automate) [How to Convert PDF to PPTX in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-PDF-to-PPTX-in-Power-Automate-and-Logic-Apps) [Merge PPTX Documents in Power Automate and Logic Apps](https://cloudmersive.com/videos/Merge-PPTX-Documents-in-Power-Automate-and-Logic-Apps) [How to Split PowerPoint PPTX Presentations into Separate Slides in Power Automate](https://cloudmersive.com/videos/How-to-Split-PowerPoint-PPTX-Presentations-into-Separate-Slides-in-Power-Automate)  
  
  
  
[Create, Edit and Convert your Microsoft DOCX Documents in Power Automate](/blog/Create%2c-Edit-and-Convert-your-Microsoft-DOCX-Documents-in-Power-Automate)  
8/18/2023 - Brian O'Neill  
  
  
![Typing with documents hovering](https://docs.cloudmersive.com/api/image/blog-image/3838c933-1636-47c9-9464-ad71230d9635) Microsoft’s dynamic DOCX format is extraordinarily popular among personal and professional content creators alike. The wide range of useful features available in a DOCX document make it possible to create articles, contracts, invoices, and even interactive forms more easily than ever before. The convenience doesn’t end there, either. With the help of Power Automate flows, we can interact programmatically with our DOCX content. We can simplify the process of carrying out routine actions like converting our DOCX files to and from PDF format, generating and editing entirely new DOCX files, extracting important data from within our existing DOCX files, and more. ![Doc Conversion DOCX](https://docs.cloudmersive.com/api/image/blog-image/6f9e29dd-dd25-4bfd-8edc-8709321f026c) Cloudmersive API Connectors on Power Automate make it easy to realize your DOCX process automation goals. Check out some of our documentation videos to get started. [Convert Word Documents to PDF in Power Automate](https://cloudmersive.com/videos/Convert-Word-Documents-to-PDF-in-Power-Automate) [How to Convert PDF to DOCX in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-PDF-to-DOCX-in-Power-Automate-and-Logic-Apps) [How to Convert a Rasterized PDF to a DOCX File in Power Automate](https://cloudmersive.com/videos/How-to-Convert-a-Rasterized-PDF-to-a-DOCX-File-in-Power-Automate) [Merge Word DOCX Documents in Power Automate and Logic Apps](https://cloudmersive.com/videos/Merge-Word-DOCX-Documents-in-Power-Automate-and-Logic-Apps) [How to Compare Two Word DOCX Files in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Compare-Two-Word-DOCX-Files-in-Power-Automate-and-Logic-Apps) [How to Get Comments from a DOCX Document (Hierarchically) in Power Automate](https://cloudmersive.com/videos/How-to-Get-Comments-from-a-DOCX-Document-\(Hierarchically\)-in-Power-Automate) [How to Convert Microsoft Word Doc (1997-2003) Files to Microsoft Word DOCX in Power Automate](https://cloudmersive.com/videos/How-to-Convert-Microsoft-Word-Doc-\(1997---2003\)-File-to-Microsoft-Word-DOCX-in-Power-Automate) [How to Create, Edit a Microsoft Word DOCX Document using Power Automate](https://cloudmersive.com/videos/How-to-Create%2c-Edit-a-Microsoft-Word-DOCX-Document-using-Power-Automate) [How to Remove Headers and Footers from a Word DOCX Document using Power Automate](https://cloudmersive.com/videos/How-to-Remove-Headers-and-Footers-from-a-Word-DOCX-Document-using-Power-Automate) [How to Replace a DOCX Text String using Power Automate](https://cloudmersive.com/videos/How-to-Replace-a-DOCX-Text-String-using-Power-Automate) [How to Validate a DOCX File in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Validate-a-DOCX-File-in-Power-Automate-and-Logic-Apps) [How to Convert HTML to DOCX in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-HTML-to-DOCX-in-Power-Automate-and-Logic-Apps) For more information on Cloudmersive API Connectors, please do not hesitate to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Scan Files for Virus and Malware Threats with Low-Code and No-Code Solutions](/blog/Scan-Files-for-Virus-and-Malware-Threats-with-Low-Code-and-No-Code-Solutions)  
8/7/2023 - Brian O'Neill  
  
  
![Lock in center of doc system graphic](https://docs.cloudmersive.com/api/image/blog-image/f06d1289-eba1-4aa1-9974-0d28385eb2fc) As file sharing and file storage continue to grow at unprecedented rates, one thing is for certain: cyberattacks are growing right alongside them. As a result, redundancy is a critical consideration in modern security architecture, and it can be accomplished in a variety of ways. The **Cloudmersive Virus Scan API** can be utilized to incorporate security redundancy in myriad workflows via no-code and low-code solutions. Check out some of our existing video documentation for tips and demonstrations spanning Nintex, Power Automate/Logic Apps, Azure Blob, AWS S3, Node.js & PHP. ![Virus scan Swagger ](https://docs.cloudmersive.com/api/image/blog-image/3fae3cbd-c962-4863-b0f8-2e92b98817e9) [How to Scan OneDrive Files for Viruses in Nintex](https://cloudmersive.com/videos/How-to-Scan-OneDrive-Files-for-Viruses-in-Nintex) [How to Scan Image Files for Viruses and Malware in Power Automate](https://cloudmersive.com/videos/How-to-Scan-Image-Files-for-Viruses-and-Malware-in-Power-Automate) [How to Scan a File for Viruses, Malware, and Non-Malware Content Threats in Power Automate](https://cloudmersive.com/videos/How-to-Scan-a-File-for-Viruses%2c-Malware%2c-and-Non-Malware-Content-Threats-in-Power-Automate) [How to Create a Custom Cloudmersive Advanced Virus Scan Connector in Power Automate](https://cloudmersive.com/videos/How-to-Create-a-Custom-Cloudmersive-Advanced-Virus-Scan-Connector-in-Power-Automate) [How to Scan SharePoint List Item Attachments for Viruses in Power Automate](https://cloudmersive.com/videos/How-to-Scan-SharePoint-List-Item-Attachments-for-Viruses-in-Power-Automate) [How to Scan AWS S3 Files for Viruses](https://cloudmersive.com/videos/How-to-Scan-AWS-S3-Files-for-Viruses) [How to Scan an Azure Blob for Viruses](https://cloudmersive.com/videos/How-to-Scan-an-Azure-Blob-for-Viruses) [How to Virus Scan and Quarantine a File in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Virus-Scan-and-Quarantine-a-File-in-Power-Automate-and-Logic-Apps) [How to Scan a Website or URL for Viruses in Power Automate and Logic Apps](https://cloudmersive.com/videos/Scan-a-Website-or-URL-for-Viruses-in-Power-Automate-and-Logic-Apps) [How to Scan Files for Viruses in Node.js](https://cloudmersive.com/videos/How-to-Scan-Files-for-Viruses-in-Node.JS) [How to Scan New Files for Viruses in Azure Blob using Azure Logic Apps](https://cloudmersive.com/videos/Scan-New-Files-for-Viruses-in-Azure-Blob-using-Azure-Logic-Apps) [How to Virus Scan File Uploads in Mendix](https://cloudmersive.com/videos/How-to-Virus-Scan-File-Uploads-in-Mendix) [How to Create a Virus Scan Wrapper API in Logic Apps and Power Automate](https://cloudmersive.com/videos/Create-a-Virus-Scan-Wrapper-API-in-Logic-Apps-and-Power-Automate) [How to Perform a Virus Scan Upload in PHP](https://cloudmersive.com/videos/How-to-Perform-a-Virus-Scan-Upload-in-PHP) ![Virus Scan Connector All](https://docs.cloudmersive.com/api/image/blog-image/325bd0e7-601b-4aa3-9a78-80812352a80a)  
  
  
  
[Make the Most of your Excel Spreadsheets in Power Automate ](/blog/Make-the-Most-of-your-Excel-Spreadsheets-in-Power-Automate-)  
8/4/2023 - Brian O'Neill  
  
  
![Spreadsheets galore](https://docs.cloudmersive.com/api/image/blog-image/49de0f70-84d8-43ff-8fda-3220bbe10300) With the help of Cloudmersive API Connectors in Power Automate, it’s possible to build powerful workflows that share Excel content across file storage systems, convert spreadsheet data into more convenient formats, and even assemble entirely new Excel files from scratch. ![Doc Conversion XLSX](https://docs.cloudmersive.com/api/image/blog-image/04c19156-22a8-4efc-ac79-65016f9f5467) To get started automating workflows for your Excel files, take a look through some of our documentation videos which demonstrate Excel-based actions in Power Automate. [How to Split Excel XLSX Files into Separate Worksheets in Power Automate](https://cloudmersive.com/videos/How-to-Split-Excel-XLSX-Files-into-Separate-Worksheets-in-Power-Automate) [How to Get All Worksheets from an Excel XLSX File in Power Automate](https://cloudmersive.com/videos/How-to-Get-All-Worksheets-from-an-Excel-XLSX-File-in-Power-Automate) [How to Convert Excel XLS (1997 - 2003) to CSV in Power Automate](https://cloudmersive.com/videos/How-to-Convert-Excel-XLS-\(1997---2003\)-to-CSV-in-Power-Automate) [How to Convert CSV to XLSX in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-CSV-to-XLSX-in-Power-Automate-and-Logic-Apps) [How to Convert XLSX to CSV in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-XLSX-to-CSV-in-Power-Automate-and-Logic-Apps) [How to Create an Excel File in Power Automate: Part 1](https://cloudmersive.com/videos/How-to-Create-an-Excel-File-in-Power-Automate-%7c-Part-1) [How to Create and Update an Excel File in Power Automate: Part 2](https://cloudmersive.com/videos/How-to-Create-and-Update-an-Excel-File-in-Power-Automate-%7c-Part-2) [How to Convert an Excel File to JSON in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-an-Excel-File-to-JSON-in-Power-Automate-and-Logic-Apps) [How to Get Rows and Cells from an Excel Worksheet in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Get-Rows-and-Cells-from-an-Excel-Worksheet-in-Power-Automate-and-Logic-Apps) [How to Convert a Single Excel Worksheet to PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-a-Single-Excel-Worksheet-to-PDF-in-Power-Automate-and-Logic-Apps) [How to Convert an Excel Spreadsheet to PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-an-Excel-Spreadsheet-to-PDF-in-Power-Automate-and-Logic-Apps)  
  
  
  
[Getting Started with Cloudmersive Connectors on Nintex](/blog/Getting-Started-with-Cloudmersive-Connectors-on-Nintex)  
7/17/2023 - Brian O'Neill  
  
  
![workflow automation](https://docs.cloudmersive.com/api/image/blog-image/d704a352-d7fe-4dd3-84c0-a1e4fbfb7c11) Calling all Nintex workflow automation customers! There are more than a dozen Cloudmersive API Connectors available on the Nintex Automation Cloud, allowing you to easily convert important files between common formats, get exchange rates and up-to-date currency values, perform powerful virus and malware scans, and much more. To get up and running in minutes, check out one of our recent "How To" videos demonstrating where to find and how to utilize Cloudmersive Connectors on Nintex. [How to Scan OneDrive Files for Viruses in Nintex](https://cloudmersive.com/videos/How-to-Scan-OneDrive-Files-for-Viruses-in-Nintex) [How to Convert Documents to PDF in Nintex](https://cloudmersive.com/videos/How-to-Convert-Documents-to-PDF-in-Nintex) [How to Convert Excel XLSX to JSON in Nintex](https://cloudmersive.com/videos/How-to-Convert-Excel-XLSX-to-JSON-in-Nintex) [How to Convert CSV to JSON in Nintex](https://cloudmersive.com/videos/How-to-Convert-CSV-to-JSON-in-Nintex) [How to Create a PDF Watermark in Nintex](https://cloudmersive.com/videos/How-to-Create-a-PDF-Watermark-in-Nintex) [How to Convert USD to Euro in a Nintex Form](https://cloudmersive.com/videos/How-to-Convert-USD-to-Euro-in-a-Nintex-Form)  
  
  
  
[Detect Server-Side Request Forgery (SSRF) Threats with Minimal Code](/blog/Detect-Server-Side-Request-Forgery-\(SSRF\)-Threats-with-Minimal-Code)  
7/13/2023 - Brian O'Neill  
  
  
Web application security should never take a backseat. If we don’t closely monitor the way our applications validate user-provided data, we can leave them vulnerable to SSRF (Server-Side Request Forgery) threats, which can be used to initiate damaging cyberattacks. ![Lock on Laptop](https://docs.cloudmersive.com/api/image/blog-image/49b4c112-4f65-47d6-9518-4f10c480dfb5) When we say that our application isn’t validating user input properly, we mean that the application is blindly accessing insecure external resources based on user-supplied instructions. Threat actors can initiate SSRF attacks by supplying a URL to our web application that references a malicious resource, ultimately placing them in the driver’s seat to control subsequent requests made by the application. Successful SSRF attacks can steal extremely sensitive data related to our server configuration or other important network resources, and they’re also one of many ways attackers can launch DoS (Denial of Service) attacks.

#### Cloudmersive SSRF Threat Detection API

Thankfully, SSRF threats can be detected and averted by carefully analyzing user-supplied URL input. Our SSRF Threat Detection API is designed specifically to detect threatening URLs from a user input string, identifying whether the URL contents are intended to compromise our web application. The API response provides a Boolean (**CleanURL** = _True_ or _False_) along with a string describing the threat level of that URL. Just like any Cloudmersive API, you can find code SSRF Threat Detection code examples in a variety of common programming languages available through your account management page. You can implement a powerful anti-SSRF threat policy in minutes, authorizing requests with your universal API key.  
  
  
  
[Two Cloudmersive APIs to Detect Insecure Deserialization Threats](/blog/Two-Cloudmersive-APIs-to-Detect-Insecure-Deserialization-Threats)  
7/7/2023 - Brian O'Neill  
  
  
Any time data travels between applications, across networks, or even within the confines of a single application, that data is converted to and transported as a stream of bytes in a process called Serialization. When that data reaches its destination, the exact opposite process occurs; the parser receiving the data interprets the incoming stream of bytes and uses that information to construct a replica of the original data objects. This follow-up process, called Deserialization, is what makes it possible for an application’s code to efficiently process and use external data within its code base. ![skull made of data](https://docs.cloudmersive.com/api/image/blog-image/6b5448bb-aedf-44a7-8f89-9e3f7aa4da40)

#### Understanding Insecure Deserialization Threats

The data parsing mechanisms responsible for deserialization can be viewed as omnipresent doorways into any application. As a result, securing those doorways is critical to any application’s security. If a threat actor successfully identifies a vulnerable data parser (e.g., one which blindly trusts and fails to sanitize user input), they can manipulate serialized objects in a way that tricks the application into taking malicious actions on their behalf. This vulnerability can be used to achieve remote code execution, force an escalation of privileges within the application, or even provide the attacker with direct access to unauthorized data.

#### Detecting Insecure Deserialization Threats

Thankfully, Cloudmersive APIs are equipped to detect insecure deserialization threats in several forms. Below, we’ll highlight our anti-Insecure Deserialization policies and discuss how they can protect your applications from harm. **Detect Insecure Deserialization JSON Attacks from Text Input** JSON is one of the most common data formats used across the digital world today. While it’s generally considered a very secure format due to its readability and relatively simplistic structure, it’s still possible for threat actors to launch insecure deserialization attacks through poorly sanitized JSON parsers, and the contemporary popularity of JSON as a safer alternative to extensible formats like XML makes it increasingly worthwhile for threat actors to try. This iteration of our Security API is designed specifically to identify Insecure Deserialization attempts within JSON strings, and it can be used to block threats before they reach the application codebase. The API response provides a Boolean value indicating if a given text string contained an attack. **Advanced Scan a File for Viruses** The Advanced Scan iteration of our Virus Scan API is designed to block multiple unique threat types in a single comprehensive policy. It allows developers and/or administrators to set custom rules against threats embedded within common file types, while simultaneously checking files for more than 17 million virus and malware signatures. One of the custom threat rules available in this API’s request body targets Insecure Deserialization threats found in JSON and myriad additional object serialization files, and you can set this threat rule to “False” in order to trigger a “CleanResult: False” response whenever Insecure Deserialization attacks are detected.  
  
  
  
[Three APIs to Check Text Input for XXE (XML External Entity) Attacks ](/blog/Three-APIs-to-Check-Text-Input-for-XXE-\(XML-External-Entity\)-Attacks-)  
6/30/2023 - Brian O'Neill  
  
  
![for xxe article](https://docs.cloudmersive.com/api/image/blog-image/1abae9bf-3b14-445c-b1c9-f572a088a5ef)

#### Understanding XXE (XML External Entity) Attacks

The advantages and risks of utilizing data in XML format are both attributable to its extensibility (it’s in the name, after all) relative to more simplistic plain-text data formats like JSON or CSV. Perhaps the strongest example of this extensibility is the XML entity declaration feature, which is defined using Document Type Definition syntax and effectively instructs data parsers to access information made available inside (internal reference) or outside (external reference) of the original XML document by the document creator. This powerful reusability feature affords numerous benefits, but it also represents a unique threat for any application parsing XML data. While XML internal entities access content stored locally within a document, XML external entities are designed to retrieve content that lives outside of an XML document, often using HTTP requests to access that information. When an XML parser reads an external entity reference, it will retrieve the information associated with that reference and subsequently incorporate that information into the original XML document. This external reference feature is a well-documented cyberattack vector. Threat actors can leverage external entity vulnerabilities to carry out XML External Entity (XXE) attacks, exploiting poorly sanitized XML data parsers with built-in document references to external malicious content. That content can take myriad forms, and the outcomes of a successful attack can be disastrous, resulting in the theft of valuable data from our servers or even Denial of Service to our users.

#### Preventing XXE Attacks with Cloudmersive APIs

Thankfully, you can take advantage of three separate Cloudmersive APIs to protect your applications from XXE threats, with each option varying in the scope of its threat detection capabilities. These options include the following:

  1. **Protect Text Input from XML External Entity Attacks** : This iteration of the Cloudmersive Security API is designed exclusively to detect XML External Entity attacks from a string of XML text. When XXE threats are identified, the API will return a Boolean labeled “ContainedXxe” to notify you if external references were present.
  2. **Automatically Detect Threats in an Input String** : This dynamic iteration of the Cloudmersive Security API is designed to detect a variety of common content threat types in a single request from text string input. These threat types include XML External Entities, Cross-Site Scripting, SQL Injection, Server-Side Request Forgery, and JSON Insecure Deserialization. When XXE threats are detected, the API response will return a “True” value in the “ContainedXxeThreat” Boolean.
  3. **Advanced Virus Scan API** : This dynamic iteration of the Cloudmersive Virus Scan API is designed to scan files for viruses, malware, AND hidden content threats, including XML external entities. The API response will determine if XML was present within any document by default; additionally, setting the “allowXmlExternalEntities” request parameter to “False” will return a “CleanResult: False” response for documents containing XML external entity references.

For more information on Cloudmersive Security and Virus Scan APIs, please feel free to [reach out to a member of our sales team](https://portal.cloudmersive.com/contact-sales).  
  
  
  
[Why OLE Object Linking and Embedding Vulnerabilities are a Serious Threat](/blog/Why-OLE-Object-Linking-and-Embedding-Vulnerabilities-are-a-Serious-Threat)  
6/16/2023 - Brian O'Neill  
  
  
As files are shared more rapidly and in greater quantities than ever before, threat actors are increasingly incentivized to exploit file sharing trends. This has organically led object linking and embedding (OLE) technology – a Microsoft document feature intended to enhance user experiences by allowing content creators to connect external applications within a document – to become a prominent cyber-attack vector. ![computer with radiating circles and binary background](https://docs.cloudmersive.com/api/image/blog-image/e3b96b42-661c-4279-96e2-bf0673a45042)

#### Why are OLE Vulnerabilities Attractive to Threat Actors?

Because OLE technology is available in several extremely common document formats (like MS word, for example), threat actors know they can reach a wide range of victims by exploiting OLE vulnerabilities. These document formats are very frequently shared within and across professional networks, increasing the likelihood that a document user will eventually activate malicious content and trigger remote code execution. In addition, links and objects embedded within a document are hard for many file scanning solutions to assess for threats, improving the likelihood that compromised documents avoid detection. Common workarounds to that problem (such as blacklisting affected file extensions) are impractical in this case given the ubiquity and importance of document formats with OLE capabilities. Further, unlike attacks involving Macros – another document vulnerability frequently exploited by threat actors – there aren’t any convenient prompts within OLE-enabled documents which give readers the option to activate or deactivate additional document features. As a result, malicious links and objects placed within seemingly normal, safe-looking documents can remain undetected for a longer period, increasing the likelihood that a document user will eventually click on a malicious object or link.

#### How are OLE Vulnerabilities Exploited to Initiate a Cyber Attack?

OLE technology allows any content creator to make external resources accessible for document readers with a convenient clickable icon. Once activated by a document user, malicious embedded objects can quickly inject malware onto the victim’s computer, connect the computer with the attacker’s server, and allow the attacker to send a disguised executable malware payload. To make matters worse, the events of an OLE attack can all happen very quickly. In some cases, the affected document viewer might not even notice the attack occurred, increasing the amount of damage the attacker can inflict before being stymied by security engineers.

#### Detect and Remove OLE Threats with the Cloudmersive Advanced Virus Scan API

The Cloudmersive Advanced Virus Scan API automatically detects OLE content within a document and returns a Boolean in the API response body indicating if OLE content was present. Administrators responsible for configuring the Advanced Virus Scan API request policies on their Cloudmersive account page (or in the API request body when using complementary code examples) can disallow files containing OLE content by setting the relevant request parameter to “False.” Once this parameter is set, any files containing OLE content will trigger a **CleanResult: False** response. This scan will also check files against a continuously updated list of more than 17 million virus and malware signatures, with coverage including ransomware, spyware, and trojans. Additional custom policies can be configured in the API request body to detect hidden non-malware threats including executables, macros, scripts, password protected files, and more. For more information about the Cloudmersive Advanced Virus Scan API, [please do not hesitate to reach out to a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[Cloudmersive API Connectors Available on Nintex Workflow Cloud](/blog/Cloudmersive-API-Connectors-Available-on-Nintex-Workflow-Cloud)  
6/8/2023 - Brian O'Neill  
  
  
On the Nintex Workflow Cloud, you can seamlessly integrate Cloudmersive APIs as value-add connector actions into your workflows. There are more than a dozen Cloudmersive API actions available in Nintex; below, we’ll cover each available action and highlight its use-case. ![cloud with connected dots](https://docs.cloudmersive.com/api/image/blog-image/c9bbdd63-09b4-4b82-a3f8-75fb6ab08f62)

#### Cloudmersive – Virus Scan

_Advanced Scan a File_  
Scan a huge variety of file types (including all office documents, text files, and over 100 image formats) with 360-degree content protection across viruses, malware, and a variety of non-malware threat types. Instantly improve your threat profile leveraging a continuously updated list of more than 17 million virus and malware signatures, with coverage including ransomware, spyware, trojans, and more. Select “Yes” or “No” from available dropdown menus to block executables, invalid files, scripts, password protected files, and macros. Restrict file types by providing a comma-separated list of acceptable file extensions (e.g., .pdf, .docx, .xlsx, etc.). Store this output in an object variable. _Scan a URL_  
Scan URLs for malicious content and threats, including viruses, phishing links, and more. Store this output in an object variable. _Scan a File_  
Perform a basic virus and malware scan on a huge variety of file types (including all office documents, text files, and more than 100 image formats) leveraging a continuously updated list of more than 17 million virus and malware signatures. Coverage includes ransomware, spyware, trojans, and more. Store this output in an object variable.

#### Cloudmersive - Convert Documents

_Convert a CSV to JSON Text_  
Quickly transition data arrays from Comma Separated Values (CSV) to JavaScript Object Notation (JSON) for more widespread application compatibility. Store this output in a text variable. _Convert an Excel (XLSX) file to JSON Text_  
Convert your Excel files to lightweight JavaScript Object Notation (JSON) strings to increase the visibility of your data. Store this output in a text variable. _Convert Document to PDF_  
Convert all major Office document file formats, myriad text files, and more than 100 image formats as PDF documents with automatic input format detection. Store this output in a file variable.

#### Cloudmersive – Currency

_Convert Currency_  
Use the “From” and “To” dropdown menus to convert a specified value between a variety of international currencies, including the US Dollar, Japanese Yen, British Pound, and more. This uses the latest exchange rate information to make your conversion. Store this output in an object variable. _Get Currencies_  
Retrieve a list of currencies available for currency conversion or exchange rate retrieval. Store this output in an object variable. _Get Exchange Rate_  
Select available currencies using the “From” and “To” dropdown menus to quickly retrieve an up-to-date exchange rate value. Store this output in an object variable.

#### Cloudmersive – PDF

_Add Text Watermark to PDF_  
Protect your intellectual property by adding a custom watermark to your PDF documents. Specify the font name, size, color, and transparency level for your watermark text. Store this output in a file variable. _Get PDF Form Field Values_  
Retrieve the names, types, and values of the form fields in your PDF documents with a single request. Store this output in an object variable. _Get PDF Metadata_  
Retrieve all the metadata stored in a PDF document, including the document title, keywords, subject, author name, creator name, modification dates, and page count. Store this output in an object variable. _Get Text from PDF_  
Convert PDF documents into plain text strings in one quick & easy step. Store this output in a text variable. _Protect a PDF_  
Secure your PDF documents by setting a reader password and owner password, and then select an encryption type from a dropdown menu (128-bit RC4 or 256-bit AES). Store this output in a file variable.  
  
  
  
[Why It's Important to Scan URLs for Malicious Content and Threats](/blog/Why-It%27s-Important-to-Scan-URLs-for-Malicious-Content-and-Threats)  
6/1/2023 - Brian O'Neill  
  
  
![Phishing Alert](https://docs.cloudmersive.com/api/image/blog-image/9ab00b9b-e475-441a-b6bb-2765e6554ee7) Malicious URLs are shared with us through a variety of mediums (including via email, file upload, and other avenues). Typically, these URLs are designed by threat actors to initiate virus downloads on our device or spoof our most familiar login portals to obtain important security credentials. In fact, the latter scenario – Phishing – represents a disproportionately large share of the world’s most active cyber security threats. According to recent reports, more than 3 billion spam emails containing Phishing links are sent out daily, which amounts to more than 1 trillion Phishing emails per year. Not all Phishing attempts are created equal, of course; some attempts are full of easily spotted errors and inconsistencies, while other attempts (i.e., Spear Phishing) are well-worded and often personally targeted to an unsettling degree. In most cases, Phishing targets don’t even see – let alone click on – spam emails containing Phishing URLs. The ones who do, however, are at a much more serious risk of clicking on the malicious URL, which immediately puts them and possibly their entire network at risk. Threat actors are playing a numbers game when they send out malicious URLs, and as a result, it’s vital that organizations take a multi-faceted approach to mitigate a consistent onslaught of URL threats. This starts, of course, with adequate user training. Companies that don’t explicitly train users to spot URL threats – especially spam Phishing and Spear Phishing attempts – leave their doors open to catastrophic data loss, data theft, virus installation, and myriad additional consequences. It’s also critical that organizations apply active security policies against URL threats. The Scan Website iteration of the Cloudmersive Virus Scan API provides a powerful solution, peeking “under the hood” of URL strings to render and analyze the link for potential threats. The API labels malicious links with a **CleanResult: False** Boolean in the API response body, making it easy to categorically delete these URLs before they can reach a compromising location within a system. In these cases, the API response will also identify the type of threat it found by name.  
  
  
  
[Restrict File Upload Types to Limit your Threat Profile](/blog/Restrict-File-Upload-Types-to-Limit-your-Threat-Profile)  
5/18/2023 - Brian O'Neill  
  
  
![uploading](https://docs.cloudmersive.com/api/image/blog-image/ddfa84fa-522e-48f8-b9ca-ebe4b0d0e378)

#### Why Limit File Upload Types?

The availability and affordability of scalable Cloud Storage solutions has increasingly made direct file upload processes more viable for businesses of all sizes around the world. The benefits of incorporating such processes are clearcut: allowing client-side users to rapidly share things like resumes, insurance claims, photos, and support documentation directly through a file upload portal into a dedicated storage instance simultaneously streamlines business efficiency and improves users' experiences. However, laying out a direct path between a storage instances and external users also opens the door to some new and daunting security challenges. This makes it possible for threat actors (meaning malicious client-side users intending to exploit a system for some monetary gain, or merely to damage its reputation for personal reasons) to carry out cyber-attacks by weaponizing file uploads in a variety of subtle ways. Many common file formats can be injected with viruses, malware, scripts, macros, and other forms of non-malware threats, and if file upload security policies aren’t carefully evaluated, monitored, and regularly updated to detect these threats, perpetrators can rapidly steal data from a system or compromise access to it entirely. The growing prevalence of file upload threats makes it crucial for businesses with direct file upload processes to implement powerful, exhaustive policies in defense of their sensitive storage instances - as well as in defense of their trusted client-side users' environments. That starts with deploying anti-virus and anti-malware policies at critical points around a network, but it doesn’t end there. It’s just as important to include rigorous content verification policies that explicitly ignore (i.e., don’t trust) file extensions and headers when validating file upload contents. Considering there are dozens of viable file formats which client-side users can choose before uploading their content for any given business service, however, broad scale content verification can appear to force an uncomfortable choice between upload security and upload efficiency. Thankfully, there's a simple and balanced solution to this problem. Rather than tackle the problem of validating/verifying dozens upon dozens of unique file formats head-on, it's best to limit the scope of the problem by disallowing unnecessary file types entirely. If, for example, a SaaS website implements a file upload process allowing customers to provide photo documentation alongside their application support tickets, there’s virtually no need to expand the list of viable image formats beyond .PNG or .JPG. Logically, most screenshots are automatically stored in .PNG format anyway, so allowing more complex formats (like .BMP or .TIFF, for example) into the fold is unnecessary to begin with. Once viable file formats are limited to a short list, the process of validating and verifying each new file’s contents becomes far more efficient and dependable, all without having any adverse impact on users' experiences.

#### Set Custom File Type Restrictions via the Cloudmersive Advanced Virus Scan API

You can easily set custom restrictions on file upload types using the Cloudmersive Advanced Virus Scan API. The **restrictFileTypes** policy allows you to provide a comma-separated list of file formats which are acceptable for your specific workflow’s needs. Once this policy is configured, all file uploads will receive full, in-depth content verification against this list of viable extensions, and any files NOT matching the extensions provided on the list will receive a **CleanResult: False** Boolean in the API response body. This makes it easy to rapidly block dozens of potentially dangerous file uploads and instantly improve your website’s file upload threat profile. Additionally, as its title suggests, the Cloudmersive Advanced Virus Scan API will, by default, scan all file uploads against a growing list of more than 17 million virus and malware signatures, including ransomware, spyware, trojans, and additional non-malware threats. This service leverages high-speed in-memory scanning to deliver sub-second typical response times. For more information on how the Cloudmersive Advanced Virus Scan API can impact your business, please [reach out to a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[How to Scan a File for Viruses and Malware in Power Automate](/blog/How-to-Scan-a-File-for-Viruses-and-Malware-in-Power-Automate)  
5/11/2023 - Brian O'Neill  
  
  
Did you know that you can scan any file upload for viruses and malware in Power Automate using the Cloudmersive Virus Scan Connector? All you need to do is incorporate the Cloudmersive Scan a File action into your flow diagram, and you can instantly check files against a list of more than 17 million virus and malware signatures, with protection against ransomware, spyware, trojans, and more. Like most operations in Power Automate, this action is very easy to incorporate. Below, we’ll walk through how you can find, configure, and test the Cloudmersive Virus Scan Connector’s “Scan a file for viruses” action in a simple, instant cloud flow (this flow logic works the same way in automated flows). From the flow diagram page, let’s start by adding an example file. We can click “New Step” and use a “Get File Content” action to retrieve the file we want. This file for this demonstration is located OneDrive for Business: ![1 - Get File Content](https://docs.cloudmersive.com/api/image/blog-image/50982e7b-d821-4684-9f04-dd8161a855a6) Once the "Get File Content" action is open, we can supply our file via the file picker. ![2 - Get File Content + File ID](https://docs.cloudmersive.com/api/image/blog-image/cf55f79a-30c4-4c49-b6de-4d6729164e5a) Now that we have our demonstration file within our flow, we can incorporate the Cloudmersive Virus Scan Connector action. To find the right Connector, let’s click “New Step” again and type “Cloudmersive” into the search bar. Once we do that, a list of Cloudmersive Connectors will populate below our search. ![3 - Search for Cloudmersive Virus Scan Connector](https://docs.cloudmersive.com/api/image/blog-image/3cebe157-118b-4158-bb7c-1a83f8fca5b4) From this list, we can go ahead and select the Cloudmersive Virus Scan Connector on the bottom row all the way to the right. Doing so will prompt us to provide a Cloudmersive API key for authentication. To get your API key, head to the Account Management center in your Cloudmersive account, click “View and Manage Keys,” and then copy your API key to your clipboard. If you don’t currently have an API key, you can get one for free by registering a free account (with zero commitments). Once our API key is entered, we’ll jump to the Virus Scan Connector actions list. ![4 - Virus Scan Connector Actions List](https://docs.cloudmersive.com/api/image/blog-image/7adbe04f-6c78-4caa-b4a8-74bdbbfae534) From this list, we can select the “Scan a file for viruses” action at the top. Doing so will open the connector action body, where we’ll find a prompt to include the input file we want to scan. We can configure this mandatory parameter by clicking on the “input file” field and selecting our file’s content from the Dynamic Content Window. ![5 - Scan a File for Viruses Action with File Content](https://docs.cloudmersive.com/api/image/blog-image/601349f1-0ea6-4dc3-a0fe-826518c2568e) At this point, we can save our flow, confirm our connections, and run our test file through the connector action. ![6 - Successful Flow Test Run](https://docs.cloudmersive.com/api/image/blog-image/8f1fa6e6-2231-437c-a09a-09e7d9d6575d) When we open the “Scan a file for viruses” action response body, we’ll see our **CleanResult** response in the action’s Outputs. ![7 - Scan a File for Viruses Response Body](https://docs.cloudmersive.com/api/image/blog-image/79a66dbd-de69-4a1a-818b-e9848ad0034f) We can now use the **CleanResult** response to dictate subsequent actions in our flow, such as deleting or quarantining our file if any viruses or malware are found. That’s all there is to it – now you can add a quick, easy layer of security to any flow. For more information on Cloudmersive APIs and Power Automate Connectors, please do not hesitate to [reach out to a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: IP Intelligence APIs](/blog/API-Spotlight%3a-IP-Intelligence-APIs)  
4/27/2023 - Brian O'Neill  
  
  
With access to consistent, accurate IP intelligence services, we can protect our applications from external threats while simultaneously improving our users’ experience through regional personalization. Thankfully, Cloudmersive IP Intelligence APIs make it easy to accomplish all the above with minimal code. Below, we’ll take a closer look at a few of our IP intelligence APIs which are oriented around IP threat detection and retrieval of geolocational data.

#### Get Intelligence on an IP Address

To investigate an IP address for potential threats, the first thing we should do is reference that address against a list of known-threat IPs. Our “Get Intelligence on an IP Address” API performs that check right away, ensuring the input IP address isn’t associated with any malicious activity (such as malware, phishing scams, etc.) and can’t be traced to any bots (such as those deployed for DDoS attacks). It also identifies if an IP address is a Tor Exit Node, which – while not necessarily indicating an imminent threat to our system – can still be associated with suspicious activity and should be monitored accordingly. In addition to checking for threats, this API retrieves as much geolocational information about an input IP address as can be found, including its associated country code, country name, city, latitude & longitude coordinates, and more. Useful information such as the currency code & currency name associated with the IP address’ region are also retrieved, making it easy to customize the way financial information (such as price information, for example) is displayed to users from different regions around the world.

#### Geolocate an IP Address

This API allows us to limit our IP check to ONLY return geolocational information. The underlying service retrieves the same array of locational data as the “Get Intelligence” API – complete with country code, country name, region code, region name, zip code, time zone (standard name), and latitude & longitude coordinates. This service is particularly beneficial for UX applications, as it can help personalize each individual user’s experience.

#### Geolocate an IP Address to a Street Address

While latitude and longitude coordinates allow us to plot locational data on a global map, street address data gives us a more specific understanding of where an IP address originates from (i.e., is that address a commercial or residential building? Is it active and functioning or empty and abandoned?). Calling our “Geolocate an IP Address” API returns the country code, country name, street address (string), city, region name and zip code associated with any given IP address. For more information on our IP Intelligence APIs, please feel free to [contact a member of our sales team](https://cloudmersive.com/).  
  
  
  
[API Spotlight: Date & Time APIs](/blog/API-Spotlight%3a-Date-%26-Time-APIs)  
4/20/2023 - Brian O'Neill  
  
  
For a network of cloud-hosted applications to operate effectively in conjunction with one another, it’s essential that they share a singular, standardized date and time retrieval service. Without syncing a date and time retrieval services across our systems, our time-sensitive automated services can quickly run off the rails, our new schedule inputs (i.e., online bookings and meetings) can be stored incorrectly, and our ability to account for perpetually moving targets on the calendar – like public holidays, for example – can fall by the wayside. Cloudmersive Date & Time APIs make it easy to integrate high quality, reliable date & time management services and sync them across all your applications. Below, we’ll take a closer look at three of our Date & Time API iterations and get a better understanding of how they can impact your business.

#### Get Current Date and Time

Implementing our Current Date & Time API gives all your applications access to a centrally available clock – synchronized with atomic clocks – ensuring the utmost accuracy and consistency in Date & Time retrieval. Executing a Get request returns the current date and time for your region (expressed in TimeSpan format), and it also returns Greenwich Mean Time (GMT) for a central reference value.

#### Get Public Holidays in the Specified Country and Year

Each country has its own evolving list of national public holidays. No matter where your business is located, it’s helpful to enumerate this list at the beginning of each year, and it’s essential to reference schedules against this data when planning important meetings or events over the course of the year. Structuring your request with country and current-year inputs will return a full list of your country’s public holidays. This includes the following array of information (for each):

  * **EnglishName** : the name of the holiday in English
  * **LocalName** : the local name of the holiday in English
  * **OccurenceData** : the exact date (in TimeSpan format) of the holiday for the specified year
  * **HolidayType** : the type of holiday in question
  * **Nationwide** : a string describing whether the holiday is observed across the entire nation



#### Parse a Free-Form Natural Language Date and Time String into a Date and Time

Generally speaking, the way people like to communicate date and time information is quite different from the way computer applications do. While computers benefit from communicating with standard, structured time values in TimeSpan format, people instead like to make lightweight, unstructured date and time references with context rooted in the present moment. For example, if todays' date is 4/18/2023, instead of asking if their peer is available to meet at 2023-04-19T13:00:00, a colleague would likely ask if they’re available to meet “tomorrow at 1 pm.” This service parses natural language inputs – such as in the example above – into timespan format, and it additionally returns integers describing the year, month, day, hour, minute, second, and day of the week values from in the original statement (as much information as is made available). To do so, it references the local date and time where the information is originally parsed and processed. The above example would return the following information:

  * **ParsedDateResult** : 2023-04-19T13:00:00
  * **Year** : 2023
  * **Month** : 4
  * **Day** : 21
  * **Hour** : 13
  * **Minute** : 0
  * **Second** : 0
  * **DayOfWeek** : Wednesday

For more information on Cloudmersive Date & Time APIs, please do not hesitate to [reach out to a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Image Information APIs](/blog/API-Spotlight%3a-Image-Information-APIs)  
4/14/2023 - Brian O'Neill  
  
  
Image files contain a wealth of important information within their file structure. If we’re using images for any downstream operation – whether that be related to image editing, sharing, or even digital forensics – retrieving this information is often a key first step in the process. We can retrieve image metadata, which most importantly provides structural information about the file’s visual content (such as its width, height, etc.), and we can additionally retrieve image color data, which yields information about the dominant color values stored in each pixel within the image. Cloudmersive Image Information APIs execute simple Post requests on image file uploads and return key information about them in a highly structured format. Below, I’ll dive into both Image Information APIs in a bit more detail.

#### Return the Image Metadata (Including EXIF and Resolution)

As mentioned above, the most basic and broadly useful metadata we can get from an image file is related to its "physical" dimensions (i.e., the height and width of the image). This data is critical for common operations - such as resizing images - which are often carried out to improve the loading speeds of media-rich pages on a website. Naturally, this API returns much more information than just height and width, however; it will additionally provide the following information:

  * **IsValidImage** – a Boolean declaring the validity of the image file
  * **FileFormat** – a string declaring the specific format of the file in question
  * **BitDepth** – an integer describing the number of bits used to represent each pixel in an image
  * **HasTransparency** – a Boolean declaring if the image contains transparency features
  * **ColorSpace** – a string describing the range of colors in the image
  * **ExifProfileName** – a string declaring the name of the Exif Profile (if available)
  * **ExifValues** – an array of Exif data (if available) including the Tag (string), DataType (string), and DataValue (string)

Please note that Exif data can be easily removed from an image by its original creator, so this information may not be available in every request. Each request for this API requires only the input image file for the operation; common file formats such as PNG, JPG are supported.

#### Return the Dominant Colors of the Image

Each color image contains a wide range of color data in its pixels, and there are always certain colors which are more prevalent than others. This API simply retrieves the top 5 dominant colors from an uploaded image and returns them in a **DominantColors** data array. This operation is especially useful if you’re trying to invert or negate the colors in an image to create a special effect (or reduce file size). Each input request should contain a valid image file in a common format such as PNG or JPG. For more information on Cloudmersive Image Information APIs, please do not hesitate to [reach out to a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Scan Cloud Storage for Viruses, Malware and Content Threats](/blog/API-Spotlight%3a-Scan-Cloud-Storage-for-Viruses%2c-Malware-and-Content-Threats)  
4/10/2023 - Brian O'Neill  
  
  
Implementing direct file upload features for client-facing web applications is becoming an increasingly popular trend among businesses all over the world. Enabled largely by the availability of affordable, scalable cloud storage solutions, direct file upload features create a linear path between client-side users and the services they seek out online, improving user experience for customers and increasing workflow efficiency for businesses in one fell swoop. It's not all sunshine and rainbows, however. Direct file uploads also open the door to a variety of file upload threats which, if not detected and contained properly, can have severe consequences for both businesses and the customers who entrust them with sensitive information. As such, it's critically important that file uploads are routinely scanned for viruses, malware, and other content-based threats at multiple points in any business' system architecture - including the cloud storage instances they reside in and are eventually retrieved from. Cloudmersive Cloud Storage Virus Scanning APIs provide powerful protection for your cloud storage instances, integrating seamlessly with Azure Blob, AWS S3, SharePoint Online Site Drive, and Google Cloud (GCP) Storage. Using minimal code to establish your connection, you can quickly gain 360-degree content protection against malicious file uploads, safeguarding your cloud storage instances against tens of millions of virus and malware signatures. You can additionally manage a variety of customizable threat rules in your API request body to block (or allow) executables, invalid files, scripts, macros & more, and you can easily restrict unwanted file types with full content verification. Below, we’ll walk through each Cloud Storage Virus Scanning iteration and highlight the specific information you’ll need to configure your request for each supported storage instance. After acquiring the necessary details, you can easily structure your API calls with your preferred choice of complementary, ready-to-run code examples - all of which are available via the “Docs & Examples” page in your Cloudmersive account.

#### How to Protect Azure Blob Storage

To configure virus scanning for Azure Blob Storage, start by gathering the following information:

  1. **Connection String** : you can obtain this information by navigating to the “Access Keys” tab of the “Storage Account Blade” in the Azure Portal.
  2. **Container Name** : provide the name of the specific Blob container you’d like to protect (within the Azure Blob Storage account).
  3. **Blob Path** : provide the path to the blob within the container; if the path contains Unicode characters, you’ll need to base64 encode it and prepend with ‘base64’



#### How to Protect AWS S3 Storage

To configure virus scanning for your AWS S3 Storage instances, you’ll need the following information ready:

  1. **Access Key** : this can be obtained by heading to the AWS console and looking within “My Security Credentials”
  2. **Secret Key** : same as above; this can be obtained by heading to “My Security Credentials” in the AWS console.
  3. **Bucket Region** : provide the name of the exact region of your S3 bucket (e.g., “US-East-1”).
  4. **Bucket Name** : provide the name of the S3 bucket.
  5. **Key Name** : parse the name of the files in S3 that you wish to scan for viruses. You’ll need to base64 encode the key name if it contains Unicode characters and prepend it with ‘base64’.



#### How to Protect SharePoint Online Site Drive

To configure virus scanning for your SharePoint Online Site Drive instances, prepare the following information:

  1. **Client ID & Client Secret**: you’ll need to follow a few steps to find this information: 
    1. Navigate to the Azure Portal and click on Azure Active Directory
    2. Click on App Registrations (left-hand side)
    3. Click on Register Application
    4. Name the application CloudmersiveAntiVirus; then click “Register”
    5. Get the client ID by clicking on Overview and copying the value labeled Application (client) ID
    6. Click on Certificates and Secrets
    7. Click on New Client Secret; choose a longer expiration and give the secret a name
    8. Copy the secret value to the clipboard; save it securely (this is your Client Secret)
    9. Begin granting permissions to SharePoint; first slick on API permissions on the left hand side
    10. Click “Add Permission” and choose Microsoft Graph
    11. Click on “Application Permissions”
    12. Search for “Site.FullControl.All”
    13. Click on “Add Permissions”
    14. Head back to Azure Active Directory; click on “Enterprise Applications”; click on "CloudmersiveAntiVirus” and click on “Permissions”
    15. Click on “Grant Admin Consent”
  2. **SharePoint Domain Name** : provide your SharePoint Online domain name (e.g., mydomain.sharepoint.com)
  3. **Site ID** : provide the ID of the SharePoint site you want to retrieve the file from
  4. **Tenant ID** : (Optional) provide the Tenant ID of your Azure Active Directory
  5. **File Path** : path to the file(s) you want to scan
  6. **Item ID** : provide your SharePoint itemID (such as DriveItem ID)



#### How to Protect Google Cloud Platform Storage

To configure virus scanning for your Google Cloud Platform Storage instances, prepare the following information:

  1. **Bucket Name** : provide the name of the bucket you’d like to protect in GCP Storage
  2. **Object Name** : provide the name of the object (or file) in GCP Storage; if it contains Unicode characters, prepare to base64 encode the object name and prepend with ‘base64’
  3. **JSON Credential File** : provide the service account credential for Google Cloud (stored in a JSON File)

For more information on how you can leverage our low-code (and no-code) Scan Cloud Storage APIs, please do not hesitate to [contact a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: PDF Annotation APIs](/blog/API-Spotlight%3a-PDF-Annotation-APIs)  
4/6/2023 - Brian O'Neill  
  
  
It’s no exaggeration to call PDF one of the most popular file types in the entire world. According to recent studies, it’s the third most common format found anywhere on the web – even ahead of JPG and PNG, which collectively store most of the internet’s image content. A subtle advantage of storing content in PDF format is the option to programmatically edit that content while it remains locked in its compressed form. Manipulating files without opening them can drastically impact the efficiency of any file sharing workflow, especially when relatively small changes – such as the addition/removal of comments or annotations in a document – are required. For developers and administrators working in industries where contracts, waivers, insurance forms, mockups, etc. are being moved from one storage instance to another in large quantities, programmatically handling document annotations in large batches efficiently consolidates precious time and resources. Our PDF Annotation APIs make it extremely easy to get, change, and remove annotations and comments from PDF documents without ever having to open those documents in the first place. Minimal code is required to structure each API call, and simple request parameters make it easy to customize new content by providing exact locational details for each addition to your documents. Below, we’ll take a closer look at each of our four PDF Annotation APIs and the way they can each be used to impact your PDF editing workflow.

#### Add One or More PDF Annotations, Comments in the PDF Document

When creating a new annotation or comment in a PDF, a variety of important information is required at the outset to define the body of the new content and its specific location within the document. This API iteration allows you to structure a single request determining all relevant details for new annotations or comments at once. Each request can be customized to define the Title, Subject, Text Contents, Annotation Index, and Annotation Type (e.g., “Text”); in addition, integers can be specified to determine the Page Number, LeftX, TopY, Width and Height specifications which will collectively place the annotation/comment at a specific point of your choosing within the document. Timespan format can also be used to set the creation and modification dates for each annotation or comment according to your preference. Once your request is submitted, the underlying service will simply return the encoding for the modified PDF file.

#### Get PDF Annotations, Including Comments in the Document

Before taking action to remove or edit existing comments and annotations within a PDF file, it’s highly beneficial to first retrieve details about their index, their location within the document, and their exact contents. Calling the “Get PDF Annotations” API iteration will automatically retrieve this information, structured in a similar way to the “Add One or More PDF Annotations” input request.

#### Remove a Specific PDF Annotation, Comment in the Document

After retrieving data about all the comments or annotations within a document, it’s possible to remove specific annotations individually using the annotation index. In its request parameters, this API iteration requires the input file path along with the annotation index(es) for removal, and it subsequently returns the encoding for the PDF File with the specified annotations or comments removed.

#### Remove All PDF Annotations, Including Comments in the Document

If your goal is to indiscriminately clear a document of all its annotations or comments, there’s no need to retrieve details about each one of them first. You can structure your request to our “Remove All PDF Annotations” iteration with your file path alone, and the underlying service will return your document with all annotations and comments removed. For more information on our PDF Annotation (or PDF Editing) APIs, please do not hesitate to [contact a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Validate Phone Number Syntax](/blog/API-Spotlight%3a-Validate-Phone-Number-Syntax)  
3/22/2023 - Brian O'Neill  
  
  
Invalid syntax can instantly turn promising contact data into a string of useless text. When it comes to phone numbers, syntax errors are particularly easy for the human eye to miss; the difference between correctly and incorrectly formatted strings often comes down to one small digit in the wrong place (or a digit missing from the equation entirely). Programmatic data validation is key to ensuring you end up with high-quality contact data. If your applications can identify problems independently upon accepting new information, you can avoid wasting precious time correcting common errors at a later date. You may not have a choice in the matter, either; if you're working in any highly regulated industry, you may be forced to comply with stringent data accuracy standards, making precise validation measures obligatory features in our workflows. Regardless of your use case, our "Validate Phone Numbers" API can make a big impact.

#### Cloudmersive Phone Number Validation API

The “Validate Phone Numbers” iteration of our Validation API can be quickly and easily deployed to ensure the correctness of new contact information within your applications. The underlying service will first validate the syntax of each phone number and, if valid, return a variety of useful information about them. This information includes the following:

  1. **“IsValid”** \- a Boolean determining the validity of the input string.
  2. **“Successful”** \- A Boolean describing the success of the operation.
  3. **“PhoneNumberType”** \- A string identifying the phone number type.
  4. **“E164Format”** \- A string displaying the phone number in E164 format.
  5. **“InternationalFormat”** \- A string displaying the phone number in international format.
  6. **“NationalFormat”** \- A string displacing the phone number in national format.
  7. **“CountryCode”** \- A string identifying the associated country code on its own.
  8. **“Country Name”** \- A string identifying the associated country name on its own.

For more information on how you can take advantage of our “Validate Phone Numbers” API, please do not hesitate to [contact a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Validate Email Addresses](/blog/API-Spotlight%3a-Validate-Email-Addresses)  
3/20/2023 - Brian O'Neill  
  
  
![fast typing hands](https://docs.cloudmersive.com/api/image/blog-image/21964f57-0ed0-470c-9db4-77c0805f9f26) Quality contact information is critically important to any business. With an organized database of email addresses, you can establish consistent, long-term communication patterns with your clients/customers and maintain positive relationships with them well into the future. As with most forms of text data, it’s essential to make sure email syntax is accurate right when it’s entered your system. Otherwise, you risk losing precious contact information to predictable typos and sloppy copy/paste jobs. In addition, it’s highly beneficial to get up-to-date information on each address’ email server right off the bat; that way, you can avoid bounce-backs when messaging each recipient in the future. Thankfully, it’s quick and easy to implement a rigorous email validation workflow with Cloudmersive Email Validation APIs. We have three separate Email Validation API iterations you can choose from, each differing slightly in breadth to serve varying degrees of need. Below, we’ll take a closer look at all three of our Email Validation API iterations and discuss their respective use-cases.

#### Fully Validate an Email Address

As the name suggests, this iteration presents our most comprehensive, multi-step email validation solution. Full email validation entails checking each email for syntactic correctness (i.e., is the email spelled & formed properly?) and subsequently identifying the mail server in question (if any). Once those steps are complete, the final step entails contacting each address’ email server to validate whether the accounts exist - without ever sending those servers an email in the process. This iteration returns strings identifying both the mail server (used for validation) and the email domain. In addition, each response supplies Boolean strings clearly stating whether the address, syntax, domain, or SMTP are valid, and whether the email address is a “catch-all domain”, a free email provider, or a disposable (limited use) email.

#### Partially Check Whether an Email Address is Valid

This iteration limits the validation process, solely identifying whether an email address’s parent domain servers are properly defined. Improperly defined servers make it difficult to contact an associated email address, and in some cases, this can even indicate malicious activity (such as email spoofing).  
In its response, this API provides one (or more) strings containing the names of defined email servers associated with the input address. It also provides an initial Boolean response describing the success of the initial operation.

#### Validate Email Address for Syntactic Correctness Only

This third and final iteration won’t attempt to contact any external servers – rather, it will simply perform a local-only check to ensure each email address is properly formatted (e.g., containing the “@” symbol and valid domain syntax). It’s a great option to use when external server responses aren’t necessary in the immediate term. Each response returns a Boolean identifying whether the address is valid, whether the email provider is recognized as a free email provider (google, yahoo, outlook, etc.), and whether the address is recognizable as a disposable address. In addition, the address domain is returned as a string. For more information on our Email Validation APIs, please feel free to [contact a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Convert Web Content to PNG, PDF & Text Formats](/blog/API-Spotlight%3a-Convert-Web-Content-to-PNG%2c-PDF-%26-Text-Formats)  
3/8/2023 - Brian O'Neill  
  
  
Uniform Resource Locators (URLs) and HypterText Markup Language (HTML) standardize the way we find and view information on the internet. Thanks to these pivotal technologies, anyone can instantly access web resources by following shareable links, and anyone with an internet browser on their device can easily view content saved in HTML format. ![Handing in a document](https://docs.cloudmersive.com/api/image/blog-image/a880837c-5b2f-48ed-bedb-cceb6fc555fd) Converting web content to non-web formats becomes necessary when we’re faced with a few common accessibility challenges. For one example, documents stored in HTML aren’t convenient to edit; those documents must be converted to a plain or rich text format when we want to efficiently make changes to them. For another, web pages don’t always load or display properly; documenting such issues requires quickly storing the present state of that web content in a static file format. Cloudmersive Web Conversion APIs make it easy and efficient to convert URL and HTML content into a variety of static, interoperable formats. With these APIs in your arsenal, you can rapidly scale your applications’ functionality and directly benefit your users, all while reducing the overhead cost of writing and implementing the code for such features yourself. Below, we’ll highlight a few of our most popular Web Conversion API iterations and discuss their respective use-cases

#### Convert Web Content to PNG, PDF

Documenting a website’s progress with static images makes a big difference in its long-term success. Since most websites undergo a variety of functional and aesthetic changes throughout their lifespan, storing each iteration of that change tells a story which sets up future web developers for success. Further, given that all websites encounter problems with loading or displaying properly at one point or another, sharing images of those problems allows support teams to solve problems quickly and effectively. There’s relevant utility for documenting external web content, too. Potentially threatening links shouldn’t be clicked on directly, so automatically rendering and documenting a suspicious URL can help avert a potential security breach. Automating web documentation with APIs streamlines the efficiency of documentation processes, eliminating the need to manually create and store images (which can vary dramatically in quality and utility to begin with). _Convert URL to PNG & Convert URL to PDF_ The URL Screenshot API fully renders a website URL and returns a PNG Screenshot of the full-page image. In your request, you can specify how much extra loading time the service should apply when rendering the website (this is useful for websites with more elements to display), and you can also specify the height and width dimensions of the resulting image. The URL to PDF API performs the same baseline function as the above, differing in that it generates a PDF from the website’s HTML code. When using this iteration, you can configure a Boolean to include background graphics from the website page and specify the scale factor for the output screenshot _Convert HTML TO PNG & Convert HTML to PDF_ The HTML to PNG API presents an alternative screenshot process to the URL iteration, rendering a website from an HTML string and returning a PNG screenshot of the website’s contents. This iteration also supports specifying extra loading weight, screenshot height and screenshot width in your request. The HTML to PDF API performs the exact same function as the URL to PDF iteration, this time sourcing an HTML string directly from the input request rather than distilling it from a URL string. Extra loading wait, background graphics inclusion and scale factor are also configurable in each request.

#### Convert HTML to Plain Text

Since HTML is used to format and display text elements, we need to first remove text from HTML code to review and edit it properly. Plain text (TXT) is a lightweight format with universal interoperability, so it makes a lot of sense to target this format when separating text elements from HTML code.  
The HTML to Text API will simply strip text content from HTML code and return that text as a single string. You can subsequently save and edit this text in any plain text editor with ease.

#### Convert HTML to DOCX

While interoperability isn’t the strong suit of a licensed rich text editor like Microsoft Word, built-in editing and multimedia display features still offer ideal editing flexibility for those with access to the application. Additionally, since it’s common to convert DOCX files to HTML so they can be viewed on a browser, automating conversions back to DOCX format at scale saves a lot of manual time and effort. The HTML to DOCX API accepts an HTML string as input and returns the encoding string for a DOCX file. For more information on our Web Conversion APIs, please [contact a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Convert Documents to PDF](/blog/API-Spotlight%3a-Convert-Documents-to-PDF)  
3/1/2023 - Brian O'Neill  
  
  
Creating PDFs is a repetitive office administration task - but it doesn’t have to slow you down. With our PDF document conversion APIs in your arsenal, you can quickly and securely convert common document types to and from PDF format – including Word DOCX, Excel XLSX, PowerPoint PPTX, and more. In this article, we’ll highlight a few of our most popular PDF Document Conversion APIs and discuss how they can impact your business. ![PDF Printed Documents](https://docs.cloudmersive.com/api/image/blog-image/8aac19ff-aff2-423b-95a3-68e5b71071e4)

#### Convert Between DOCX And PDF

There are few word processors more widely known and used than Microsoft Word, which means DOCX files are an exceedingly common sight on most work computers. As useful as Word may be for creating content, however, it’s not quite as ideal for sharing it, because there’s no guarantee that downstream document recipients will have Microsoft Word installed on their device. Converting DOCX documents to PDF will ensure that the original document contents – including all rich formatted text, tables, and multimedia elements – can be viewed by anyone in a standard, interoperable format while simultaneously adding a layer of document security. Conversely, if those PDF files need to be opened and edited, converting back to DOCX will do the trick. Our DOCX to PDF and PDF to DOCX conversion APIs make it easy and simple to convert between documents at scale, requiring each file path in the request and returning each output file encoding string when the operation completes.

#### Convert Between XLSX and PDF

For several decades now, Excel has dominated the market for spreadsheet applications. Presently, there are more than a billion unique Excel users worldwide - it’s no exaggeration to say that millions of professionals can’t go more than a week without opening an XLSX document for one reason or another. When it comes to publishing or sharing finalized projects created in Excel (especially reports), however, XLSX isn’t the ideal format to use. Not only will PDF format provide a more clean, presentable format for viewing spreadsheet data, but it’ll also increase overall document security as well, ensuring unwanted external users can’t access and change important information on a whim. Our XLSX to PDF and PDF to XLSX conversion APIs allow developers to automate the process of exporting key XLSX reports in PDF format and just as easily return PDF documents to XLSX format when the need arises.

#### Convert Between PPTX and PDF

While we might think of PowerPoint as a decades old slide-by-slide presentation design software, the modern, real-life uses of PPTX far exceed that parameter: PowerPoint is widely used for designing logos, video thumbnails, interactive documents, infographics and much more. However, with the amount of complex information they store, these files can become very large, which makes sharing PPTX over email and messaging applications a bit of a hassle. PDF presents a lighter-weight format for sharing (especially when rasterized) and, similarly to the DOCX use case, it helps ensure the project’s interoperability with external operating systems. Our PPTX to PDF and PDF to PPTX conversion APIs make it easy to convert PowerPoint documents to and from PDF format with a high degree of efficiency.

#### Convert Document to PDF

If you’re developing a workflow to convert multiple separate file extensions to PDF, automatic file detection may well be the solution you’re looking for. Rather than converting from an expected file extension such as DOCX, XLSX, or PPTX, an autodetection conversion API will first read the file extension for the input file and perform the correct PDF conversion based on that result. Our Convert Document to PDF API is equipped to detect all major office document formats (including DOCX, DOC, XLSX, XLS, PPTX, PPT, HTML, TXT, and more than 100 common image formats such as JPG, PNG, TIFF, etc.) from the file included in your request and will return the PDF file encoding as a string. For more information on our PDF Document Conversion APIs, please [contact a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Split Documents into Separate Files](/blog/API-Spotlight%3a-Split-Documents-into-Separate-Files)  
2/24/2023 - Brian O'Neill  
  
  
While document merging can help optimize content for short-term presentation or printing needs, it’s rarely to our advantage having large multipage files lying around in storage. For one thing, large files are more difficult to share across most networks, and for another - depending on the industry these files are used in – they can represent a violation of compliance requirements (which may compel a company to separate sensitive data to reduce the risk and impact of data breach). Document splitting and merging are similar in one respect – they are both time-consuming manual processes which benefit immensely from API automation. The Cloudmersive Convert API makes it easy to implement secure document splitting services into your file processing applications with minimal code, greatly reducing the burden of manual file processing. Below, we’ll highlight a few of our most popular Split Document API iterations. ![document binders on table](https://docs.cloudmersive.com/api/image/blog-image/3aba3a3a-7392-419f-9c59-d9be461ee481)

#### Split Word DOCX Document into Separate Documents by Page

Microsoft Word (DOCX) is widely used and depended upon for creating and storing specially formatted, rich-text documents, including anything from legal contracts to questionnaires. The Split Word DOCX API iteration is designed to automatically split an input DOCX file into separate documents – exactly one per page – and return each result alongside its corresponding page number. The API response will contain both a document URL and a DOCX file encoding string for each document, unless a “returnDocumentContents” Boolean is specified in the request to return only a file URL.

#### Split Excel XLSX into Separate Worksheets

Boasting more than a billion unique users worldwide, it’s no exaggeration to say that much of the business world depends on Excel to organize and model their data. Since XLSX spreadsheets break down the contents of each individual spreadsheet into worksheets rather than pages, the Split Excel XLSX API iteration returns each new file’s encoding relative to the name of its corresponding worksheet name and worksheet number. To return only a URL, you may set the “returnDocumentContents” boolean in your request to “false.”

#### Split PowerPoint Presentation PPTX into Separate Slides

While PowerPoint is chiefly depended upon to format and organize digital presentations, its real-world uses often exceed those parameters. For example, it’s common to use PowerPoint slides to create logos, video thumbnails, animations, storyboards, and much more. The Split PowerPoint PPTX API iteration returns one new PPTX file per slide of your original PPTX document, making it easy to separate and quickly build upon specific slide information or designs. The presentation contents and URL for each slide will be returned relative to slide number, and as with previous examples in this article, the “returnDocumentContents” Boolean may be set to “false” to return only a URL.

#### Split PDF into Separate PDF Files

Given the universal interoperability of PDF format, it’s common that multi-page DOCX, XLSX or PPTX documents are exported and shared as PDF documents. With that in mind, a PDF splitting operation effectively doubles as a DOCX, XLSX and PPTX splitting operation, ensuring the contents of each document type (and many additional document types not listed here) can convert back into their original format when the need arises. The Split PDF Document API iteration cleanly divides an input PDF document into one new file per page, returning file encoding which can easily transition back into its original file format through a simple document conversion operation (unless the PDF document is rasterized). As highlighted in the prior API descriptions, you may elect to return only a PDF URL by setting the “returnDocumentContents” Boolean to “false”. For more information on our Document Conversion APIs, please [contact a sales representative](https://cloudmersive.com/).  
  
  
  
[API Spotlight: Name Validation APIs](/blog/API-Spotlight%3a-Name-Validation-APIs)  
2/17/2023 - Brian O'Neill  
  
  
![Names on wedding table](https://docs.cloudmersive.com/api/image/blog-image/2642c1b1-492e-4981-9a2a-7466544dfa53) It’s no exaggeration to say that our names are a critically important part of our identify. The way our names are pronounced and spelled, for example, is often rooted in our own personal family history, so when others misspell or mispronounce them, it’s easy for us to feel offended or exasperated as a result. This sensitive subject intersects with online business when we ask customers to record their names and contact details on our website. When this contact information is used to automate customer outreach (such as in emails, cards, etc.), it’s extremely important that each customer’s name is spelled the way they expect it to be, or our relationship with that customer can immediately begin on the wrong foot. Names need to be entered correctly into our systems out the outset, and name validation APIs provide the logic to ensure that happens. Below, we’ll highlight some of our Name Data Validation API iterations and discuss the value they bring to any application storing customer information.

#### Validate a First or Last Name

Our two most basic name validation APIs identify whether first and last name entries are valid based on an inclusive set of criteria. Rather than rigidly referencing first and last names against a database – a practice which often fails to include many unique, personal spellings of names – the first and last name validation APIs will simply ensure that numbers, irrelevant symbols, spam inputs and blank entry fields are avoided during name entry. Common spellings of names will receive a ValidKnown result, while more rare or altogether new spellings of names will return a ValidUnknown result, ensuring customers can use a unique spelling and also quickly identify typos if that spelling was unintentional. Other name validation responses include InvalidSpam, InvalidCharacters, and InvalidEmpty.

#### Parse and Validate a Full Name

This API iteration is designed to allow users to enter full names – including suffixes, nicknames, and titles – into a single field as a string. This iteration makes it possible to capture a wide variety of information about our customers and naturally accommodate both longer family names and titled names, such as those starting with “Dr.”, “Mr.”, “Mrs.”, etc. When a full name string is entered, this API will automatically identify each element of the name and break it down into its component parts, including FirstName, LastName, MiddleName, Nickname, Title, and Suffix. Out of all these components, only the first and last names will be directly validated for accuracy, using the same validation logic & validation responses described for the previous iteration. This API will first provide a Boolean response determining the success of the operation. Additionally, after breaking a full name into its components, it will return the full DisplayName as a string.

#### Get the Gender of a First Name

Since names often reflect a person’s gender identity, storing gender information based on name inputs allows us to customize our relationship more effectively with each customer. If, for example, we are storing customer information on an ecommerce shopping website, we can use gender categorization to automatically tailor product suggestions to a customer and increase commercial engagement. It's important to note, however, that not all names directly indicate a particular gender, and implied gender can also vary based on culture. As a result, this API takes into account both first names AND country codes, and returns possible gender values including Male, Female, and Neutral. For more information on our Name Validation and Data Validation APIs, please feel free to [make an inquiry with our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Image Recognition APIs](/blog/API-Spotlight%3a-Image-Recognition-APIs)  
2/8/2023 - Brian O'Neill  
  
  
![printed photos on a table](https://docs.cloudmersive.com/api/image/blog-image/0f9b4334-9a53-4d08-ae17-232873343b9d) From digital forensics to automatic photo captioning, our image recognition APIs make it possible for you to add powerful, intelligent features to your image processing applications at low cost. Below, we’ll take a closer look at a few of our most popular Image Recognition API iterations and examine how they can be used to impact your business.

#### Generate Perceptual Hash; Calculate Similarity Between Perceptual Hashes

Perceptual hashing is a very reliable and lightweight method used for comparing images against one another. It’s frequently used in digital forensics to track down illegally disseminated images, and it’s also regularly employed to identify cases of copyright infringement, such as a scenario where some unlicensed entity may have stolen a protected image and used it online for commercial gain. Creating a perceptual hash for any image involves calculating a hash value based on that image’s specific properties, such as its color and texture. Once that hash is generated, it can be compared with the perceptual hash of a second image to identify the differences between them. The degree of difference between these two hash values is referred to as a “Hamming Distance” (named after Richard Hamming, a famous American mathematician whose research provided the basis for this operation). The “Generate Perceptual Image Hash” API iteration accepts a common image format (such as PNG, JPG) in its request and returns an image hash as a string. The “Calculate Similarity Between Two Perceptual Image Hashes” API iteration accepts two separate perceptual hash strings as input and returns an “ImageSimilarityScore” based on the Hamming Distance between them. Together, both API iterations can be used to protect your online image content or identify duplicate images within your own systems.

#### Describe an Image in Natural Language

There are dozens of unique, exciting applications of Deep Learning Artificial Intelligence (AI) which have become increasingly democratized in recent years, and automatic image description is a worth inclusion on that list. At a high level, this process involves preprocessing and extracting key features from an image which can be subsequently encoded and referenced against a training dataset. There are a variety of applications for this technology, including automatic image captioning and indexing (making images easier to search with keywords). This API accepts an image in a common format (such as PNG, JPG) and generates two separate descriptions for that image in return. The first description is the “BestOutcome,” and the second description is the “RunnerUpOutcome.” A confidence score ranging from 0.0 – 1.0 is provided alongside each description, with lower values indicting lower confidence, and higher values indicating higher confidence. Additionally, a “HighConfidence” Boolean response is provided independently of both descriptions to initially evaluate the overall success (potential utility) of the operation.

#### Detect People in an Image, Including their Location Within the Image

Whether you’re analyzing still-frames from security camera footage or planning to crop photos automatically for our website, understanding the existence and location of human subjects within your image is extremely important. Like the previous API described in this post, this operation also relies on feature extraction, leveraging a vast training dataset to accurately distinguish human forms from non-human forms within the confines a digital pixel matrix. This API will identify the position, size and location of human subjects within an image – regardless of which direction those people are facing (i.e., this does not rely on facial recognition). The specific height, width and pixel coordinate location of each detected human subject is expressed in terms of pixels, and an overall “ObjectCount” is provided to summarize the total number of people identified.

#### Detect and Unskew a Photo of a Document

When we take photos of documents on our handheld cameras and smartphones, the images we take are rarely perfect, and in many cases these imperfections make subsequent OCR or document conversion operations difficult to perform with high fidelity. As a result, when we build application workflows designed to funnel images into OCR or document conversion operations, it’s important to first incorporate a step which automatically corrects our images’ imperfections, thus optimizing their contents’ machine readability. The purpose of this API is specifically to assist in the downstream OCR and conversion operations mentioned above. This API accepts images in common formats (such as PNG, JPG) and automatically corrects natural image skewing, returning a perfectly square image. When calling this API, you’ll have the option to apply a Black and White postprocessing effect to the image as well, which serves to further aid in subsequent OCR operations. For more information on our Image Recognition APIs, please feel free to [contact our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Video Processing APIs](/blog/API-Spotlight%3a-Video-Processing-APIs)  
2/3/2023 - Brian O'Neill  
  
  
![beach video capture](https://docs.cloudmersive.com/api/image/blog-image/d6d438da-2a4d-441e-b0d5-7c7e4eb53702) Developing an efficient video upload process for any web application can be a bit challenging. That’s chiefly because videos are much larger than other forms of content, typically incorporating multiple types of media at once - which puts extra pressure on the web servers responsible for processing them. The Cloudmersive Video API contains eleven useful iterations which greatly reduce the hassle of processing video files. They can effectively offload pressure from your web application's workflow, freeing up server resources while expanding your application's functionality. Below, we’ve highlighted four of our most popular Video API iterations with context for their respective use cases.

#### Convert a Video to MP4 Format

Many people are familiar with MP4 format, and even if they aren’t, there’s a good chance they use it unwittingly when exporting their videos. There’s a good reason for MP4’s sustained popularity – it offers excellent compression and streaming capabilities (along with a host of multimedia benefits) and it continues to be widely supported as a result. There are, however, still a variety of other popular video formats out there (such as WebM and AV1), so it’s important that any website preferring MP4 files has a way of converting these other formats to MP4 when the need arises. This API automatically detects a variety of common video formats (using file path or file URL) and converts them to MP4 format. In addition to this, it provides a host of format customization options, allowing you to influence your output product in each request. When calling this API, it’s possible to change the height, width, aspect ratio, framerate, and output quality (compression) of a video during its conversion to MP4. Please note that the Video API also supports video conversions to WebM, MOV, GIF and PNG (for still frames) formats.

#### Resize a Video, Preserving OR Ignoring Aspect Ratio

There are two Cloudmersive Video API iterations which can be used to physically resize videos: one which preserves the original aspect ratio (height/width ratio), and one which ignores aspect ratio, allowing instead for full customization of video dimensions. Both iterations are important additions to any video processing workflow; the former allows us to scale a standard output file up or down, while the latter facilitates an entirely unique look and feel. When structuring an API call to either of these video resizing iterations, you’ll have the option to specify the maximum height and/or width of the output file, along with the framerate and compression level. Please note - when selecting files for these operations, any file larger than 2 GB should be included in URL form (rather than file path).

#### Cut a Video to a Shorter Length

“Short and sweet” has become a go-to trend for generating engaging video content. We’re growing more and more accustomed to consuming 5, 10, 15 or 30 second videos which jump straight to the action, fighting to capture ever-shortening attention spans. Offering a video shortening service in your application will make it easy for client-side uploaders to decide on the final length of their content without having to waste time editing and re-uploading their videos. This API iteration allows you to easily cut videos to a specified length using specific start and end times in TimeSpan format. If an end time isn’t specified, the API will cut anything before the start time and leave the remainder of the video (up to 4 hours) untouched.

#### Scan a Video for NSFW Content

For any workflow handling visual content uploads, content moderation is a key checkpoint in the upload process. It’s important not to let videos containing NSFW content (such as racy or pornographic material) reach storage, or we run the risk of inadvertently offending any client-side visitors or internal colleagues who may need to access that content downstream. Human content moderation teams can independently review small volumes of content uploads without programmatic intervention, but large volumes of uploads call for a more scalable solution to assist in that review. The NSFW Video Scanning API reviews each frame within an uploaded video, using advanced Machine Learning AI to identify racy or pornographic content. If such content is identified, this API will return invaluable information for content moderation purposes, including the total number of racy or NSFW frames in the video and the exact timestamp & frame number for each. Along with each NSFW frame, a content classification result string and numerical score is provided, helping diagnose the degree to which the flagged content may be offensive. For more information on our Video APIs, [please contact our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Security APIs for Content Threat Detection](/blog/API-Spotlight%3a-Security-APIs-for-Content-Threat-Detection)  
1/20/2023 - Brian O'Neill  
  
  
![dark lighting with typing on laptop](https://docs.cloudmersive.com/api/image/blog-image/df799e77-72cd-4eea-97d2-15565799f665) Since a company's web servers will always contain sensitive (and lucrative) information, its web applications will always present a worthwhile target for cyber criminals. When attackers are successful in their attempts to hack a company’s web servers, they can leave their victims reeling to recover losses - or put them out of business entirely. Thankfully, with Cloudmersive Content Security APIs in your arsenal, many common web application breach attempts can be deterred - all with only a few lines of code. Below, we’ve highlighted three of our most popular Security API iterations which help prevent Cross-Site Scripting (XSS), XML External Entity (XXE) attacks, and SQL Injection (SQLI) attacks respectively.

#### Cross-Site Scripting (XSS) Detection & Prevention API

The goal of a Cross-Site Scripting (XSS) attack is to steal important information from your website visitors. An attacker can accomplish this easily if a website doesn’t properly validate user inputs; they’ll inject their own code into our website, and that code will subsequently execute in the browser of an unsuspecting visitor, allowing the attacker to strip information (such as login credentials) from that visitor. The Cross-Site Scripting Detection API identifies and removes XSS attacks automatically through normalization, rendering the attackers’ actions inert and returning a normalized result string (labeled “NormalizedResult”). The information returned by this API also includes a Boolean (labeled “ContainedXss”) indicating if an XSS attack was identified, along with a string (labeled “OriginalInput”) which displays the original XSS input.

#### XML External Entity (XXE) Detection API

If your web application parses external XML entities, it’s critical to protect it against XXE attacks. Since XML format can store a variety of complex information, it’s possible to trick poorly configured XML parsers into referring to & accessing sensitive data based on malicious references hidden within an XML string. It's important to note that preventing XXE attacks starts with evaluating the parser itself: do you need to enable external entity processing in the first place, or can you configure your parser to only process data from sources you trust? In any case, it’s sensible to involve a rigorous security policy in this process. Our XXE detection API makes a huge difference in the data validation process, immediately identifying malicious references buried within XML schemas and supplying a simple Boolean response (labeled “ContainedXxe”) when an XXE attack is detected.

#### SQL Injection (SQLI) Detection API

When client-side users query servers for information, their search is often transformed into a standard SQL query which attempts to access server information on their behalf. Attackers using a SQL Injection strategy will specifically attempt to exploit this process by including malicious queries in the user input phase. If user inputs aren’t properly validated, these queries can be inadvertently processed, bypassing our security policies and giving the attacker unrestricted access to our server data. The SQLI Detection API enacts a critical security policy during the user input validation process. It provides a Boolean response (labeled “ContainedSqlInjectionAttack”) indicating if an SQLI attack was detected in a particular string. For more information about our Security APIs (and additional security products), please [contact our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Easy Photo Filter APIs ](/blog/API-Spotlight%3a-Easy-Photo-Filter-APIs-)  
1/3/2023 - Brian O'Neill  
  
  
![](https://docs.cloudmersive.com/api/image/blog-image/16633380-555b-4d2c-9b36-64411d9eda89) On websites where images are frequently uploaded, basic image filters are a great inclusion – especially with the availability of high-quality, low-code image filtering APIs to get the job done. It's become increasingly common place to find a catalogue of image filtering options available when we upload our content to social websites (such as social media, ecommerce, etc.) - yet another testament to the constant democratization of complex technologies in the digital age. Many such filtering services, such as those which allow us to adjust contrast, blur, sharpen, brighten, or even emboss our images, were once only accessible through licensed photo editing applications or widgets limited to desktop use. Thankfully, you don’t need to write dozens of lines of code to include image filters in your applications; our suite of Image Filtering APIs will allow you to easily build a variety of simple photo filtering services into any application using only a few lines of ready-to-run, copy & paste code examples. Below, we’ll take a closer look at three of our eight Image Filtering APIs and their respective benefits.

#### Gaussian Blur API

The Gaussian Blur filter gets its name from the mathematical function used to model a blur effect's distribution across the pixels in a digital image. Formulas aside, the outcome of a Gaussian Blur is a simple effect that we’re quite used to seeing in online images: certain areas within an image become blended with one another to reduce detail, making a once sharply defined pixel matrix appear slightly out of focus instead. There are a variety of useful applications of this filter, and perhaps the most common among those is to reduce noise (random variations) in an image. Performing a Gaussian Blur is also a great step to take before overlaying text on top of a photo, as it will increase the contrast between the text and the photo, making the text easier to read. Our Gaussian Blur API's request parameters include the following:

  1. **imageFile** – the image file to perform the operation on (common formats like JPEG, PNG supported)
  2. **radius** – the intended radius of the blur operation (higher integers result in a larger blur effect)
  3. **sigma** – the variance of the blur operation (higher integers result in a more greatly varied blur effect, directly impacting the Gaussian function used to generate the filter)



#### Embossment API

Embossment is a long-standing practice with roots in physical printing processes. As far back as the 1400's, publishers saw value in making three-dimensional impressions on otherwise flat surfaces; it increased contrast between a subject and its surface, resulting in a pleasant tactile element. To this day, physical embossment is still applied to books, logos, credit cards and business cards, and you might still occasionally notice embossed grooves when drinking from glass soda bottles. Digital embossment seeks to emulate this centuries-old physical effect by generating contrast in certain areas around the subjects of a photo, which creates the impression that they are raised off a 2-dimensional surface. While online digital images clearly lack a tactile element, this filter helps emulate the enhanced aesthetic which contoured objects provide. Our Emboss Image API's request parameters include the following:

  1. **imageFile** – the file to perform the operation on (common formats like JPEG, PNG are supported)
  2. **radius** – the radius of the embossment operation (expressed in pixels; higher integers result in a greater embossment effect)
  3. **sigma** – the variance of the embossment operation (higher integers result in a more varied embossment effect)



#### Grayscale API

Where color can’t adequately distinguish the finer details of a photograph, grayscale often can. Grayscale filtering – more commonly known as black-and-white filtering – is a simple process with a powerful effect. Digital images store pixels which present a variety of complex hex (color) values, and when we apply a grayscale filter, we simply remove that color information from the equation, expressing visual information in terms of brightness levels instead. This filter creates an iconic moody effect, and it’s often applied to draw our eyes to certain areas of an image (especially darker areas) which may not have initially attracted our attention. It also serves the practical purpose of reducing the overall size of an image, which makes that image easier to process (in operations such as Optical Character Recognition, for example) and store efficiently. The Grayscale API only requires the input image in its request parameters (common formats like JPEG, PNG are supported). For more information on our Image Filter APIs, please feel free to [make an inquiry with our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Data Conversion APIs](/blog/API-Spotlight%3a-Data-Conversion-APIs)  
12/12/2022 - Brian O'Neill  
  
  
Converting between data formats is often a necessary (and frustrating) step in the process of obtaining new datasets. Thankfully, though, data conversion APIs can make that process easy. The Cloudmersive Data Conversion API includes a variety of secure, powerful iterations designed to convert to and from common data formats like JSON, XML, CSV, XLSX and more. In this article, we’ll quickly take a closer look at a few of our most popular Data Conversion APIs. ![Hand pointing at data sheet with pencil](https://docs.cloudmersive.com/api/image/blog-image/5d59078f-fbbb-4e9b-b522-036cd009292e)

#### Convert XML to JSON | Convert JSON to XML

Both XML and JSON are ubiquitous text data formats used widely among the global software development community. While the relative simplicity and readability of JSON has made it a more attractive format to a growing developer community, XML still presents a considerably flexible, powerful structure, and it remains the only viable messaging format for SOAP APIs. Making conversions between these two formats is extremely common, so having APIs in place to automate that process is a big help. Our "Convert XML to JSON" API iteration will accept either an XML string or file as input and return a JSON Object in response. Two additional APIs – "Convert JSON Object to XML" and "Convert JSON String to XML" – will handle JSON Object and JSON String conversions to XML respectively.

#### Convert XLSX to JSON | Convert XLSX to XML

It’s safe to say XLSX – Excel’s unique zip-archived Open Office XML format, first introduced for Excel 2007 – is one of the most frequently used formats on earth. While it’s simple enough to point-and-click-export individual XLSX files to common data formats like XML and JSON, larger scale conversions require more powerful, programmatic solutions to get the job done. Our "Convert XLSX to JSON" and "Convert XLSX to XML" APIs both accept a standard XLSX file input, returning a JSON Object Array & XML file string respectively.

#### Convert CSV to JSON | Convert CSV to XML

For storing flat, tabular data, CSV is one of the best options around; it’s only when data relationships become more complex and hierarchical - or when compatibility with specific APIs/applications comes into question - that more robust formats (like JSON and XML) are required. Our "Convert CSV to JSON" and "Convert CSV to XML" APIs accept a CSV file as input, returning a JSON Object Array and an XML file respectively. Additionally, both offer an optional input request parameter – _columnNamesFromFirstRow_ – which allows you to specify via Boolean whether the first row of your CSV file should be used for your columns. For more information on our Data Conversion APIs, [please contact a member of our sales team](https://account.cloudmersive.com/contact-sales).  
  
  
  
[API Spotlight: Merge Your Office Documents with Ease](/blog/API-Spotlight%3a-Merge-Your-Office-Documents-with-Ease)  
12/5/2022 - Brian O'Neill  
  
  
![Overhead View of Documents](https://docs.cloudmersive.com/api/image/blog-image/31b6b5b3-69f1-4f2f-b293-148a6a40cac0) Merging content of the same file type is, at minimum, a simple way to consolidate your documents into a single file and share them more conveniently via email (or any online exchange). From a more specific, practical standpoint, it’s an important step in piecing together larger memos, presentations, and reports for which business materials may have accumulated separately over time. Available via the Cloudmersive Convert API endpoint, our Merge Document API iterations make it easy to programmatically combine either two or multiple (more than two) documents of a common file type into a single, comprehensive file. There are fourteen Merge Document API iterations in total; below, we’ll take a closer look at a few of the most frequently used services among those.

#### Merge PDF Files

Combining PDF files is a common need for office admins and content professionals everywhere. The reason for that is, in large part, because PDF is a standard destination format (easy for anyone to open & view on any platform) for documents originally created in less “friendly” applications, such as DOCX, Excel, and PowerPoint. Merging PDF files is a great way to assemble multidimensional content of almost any kind, including reports, presentations, image arrays, and more. Electing to use our “Merge Two PDFs” or “Merge Multiple PDFs” APIs will allow you to quickly combine between two and ten PDF files (respectively) at once.

#### Merge Excel Files

Merging Excel files together entails the creation of a single, new spreadsheet containing a proportionate number of separate worksheets within it. Combining spreadsheets in this manner is a highly efficient way to handle commonly occurring scenarios, such as when multiple related datasets are received and stored independently of each other over a period of time. The key value here is organization; it’s easy for high volumes of spreadsheets to clutter your file storage and create costly information gaps. You can merge up to ten Excel files at once using the “Merge Multiple Excel Files” API, or you can limit your operation to only two files using the “Merge Two Excel Files” API.

#### Merge HTML Files

Combining HTML documents is a slightly different operation than the prior two services highlighted in this post. In this case, order isn’t the only thing that matters; rather than generating a new file with multiple separate pages, this operation will stack the HTML code from each input document below the contents of the input preceding it. The result is a crucial conservation of whitespace, which is important to ensure your HTML file displays its contents correctly when opened in a web browser view. You can combine up to ten HTML documents using the “Merge Multiple HTML Files” API iteration, or you can use the “Merge Two HTML Files” API to consolidate only two.  
  
  
  
[API Spotlight: Cloudmersive Image Editing APIs](/blog/API-Spotlight%3a-Cloudmersive-Image-Editing-APIs)  
11/28/2022 - Brian O'Neill  
  
  
![Image Editing](https://docs.cloudmersive.com/api/image/blog-image/ee3cc236-163d-4575-abc7-09262ce2d000) In this post, we’re shining a spotlight on our Image Editing APIs, all of which are available via the Cloudmersive Image Recognition & Processing API endpoint. Whether you’re looking to make simple or substantial changes to your images, our Image Editing API iterations have you covered. There are fourteen Image Editing API iterations in total; in this post, we’ll highlight five commonly used iterations and briefly discuss their respective use-cases and parameters. Please note that each Image Editing API iteration accepts common file types like PNG and JPG.

#### Adaptively Adjust the Contrast in an Image

The gamma value of an image determines how easy or difficult it is to make out distinctive features within the image itself. If a photo is too dark to begin with, adjusting its gamma will help you see distinct shapes that may not have been discernable at first. This API iteration allows you to change the gamma value of any image by entering a specific (double) numerical value. Values between 0.0 and 1.0 will reduce the contrast in your image, while values above 1.0 will instead increase contrast. Given it’s much more common to increase gamma when using this operation, the default gamma value is set at 2.0.

#### Rotate an Image any Number of Degrees

An important feature of many basic image editing applications is the option to rotate images in positive (left rotation) or negative (right rotation) multiples of 90 degrees. This API iteration goes a step further, allowing you to achieve a much more specific degree of rotation by entering any (double) value between 0.0 and 360.0.

#### Add a Customizable Drop Shadow to an Image

Adding a drop shadow to an image is an easy way to give it a three-dimensional feel; it’s a method often used by marketing/content creation professionals to add depth to an image, making it more pleasant to look at. This API iteration offers you the ability to create and customize clean, engaging drop shadows for your images. When configuring this API’s parameters, you can specify the exact horizontal & vertical offset of the shadow, as well as its sigma (blur distance) and opacity (ranging from 0% to 100%).

#### Draw Text onto an Image

Whether you're creating polished marketing materials or generating viral internet memes, text and images are often inseparable. This API iteration affords you the ability to programmatically draw text onto an image by supplying plain text along with various details about your desired font, X/Y pixel coordinates (indicating exactly where the custom text should show up), and the height and width of your text box (in pixels).

#### Composite Two Images Together (Basic or Precise)

Compositing images is a common and useful way to generate a “surreal” image effect. This usually entails placing the subject of one image onto the background of another (imagine, for example, an image of a Giraffe grazing from a tree in Central Park) or fully blending two images with varying degrees of opacity. There are two iterations of Cloudmersive Image Compositing APIs available for you to use: one which performs a basic version of this operation, and another which performs a much more precise/advanced version of this operation. The basic version only requires you to specify a general composite location, such as “top-center” or “center-left,” while the precise iteration allows you to dictate the exact coordinates (in pixels) of the composite’s location. The precise iteration also allows you to control padding by including transparent pixels at the border(s) of your layered images as needed.  
  
  
  
[API Spotlight: Cloudmersive Image Conversion APIs](/blog/API-Spotlight%3a-Cloudmersive-Image-Conversion-APIs)  
11/21/2022 - Brian O'Neill  
  
  
![Browsing Images](https://docs.cloudmersive.com/api/image/blog-image/008c6e10-a1f7-4acf-b5a3-7b48ca46a394) In this article, we’ll take a closer look at our suite of Image Conversion APIs, which will quickly convert images (from a wide variety of input image formats) into seven of the most common image formats used around the digital world. These Image Conversion API iterations are all available via our Image Recognition & Processing API endpoint, and they’re all equally easy to use either through custom-code integrations (with complementary, copy-and-paste code examples available on our API Console), or as connections through Microsoft Power Automate/Logic Apps, MuleSoft, Workato, and various other enterprise automation platforms.

#### JPG & PNG Conversions

Chief among our Image Conversion API iterations are two services representing the most ubiquitous image formats found online: PNG and JPG. Employing our “Convert Image to PNG” or “Convert Image to JPG” APIs will make it easy to transition dozens of image file types into either format with ease. It’s important to note that JPG files – unlike PNG – are quite lossy, meaning they require a bit more attention to ensure your desired output image quality is achieved during your conversion. To that end, our “Convert to JPG” API offers a special parameter allowing you to specify the quality of your output JPG files. This parameter can be satisfied by supplying an integer between 1 (highest compression; lowest quality) and 100 (lowest compression; highest quality); the recommended value is 75.

#### GIF Conversions

Perhaps most known for displaying the basic, frame-by-frame animations adored by millions online, GIF format is also frequently used for a more refined, professional purpose: displaying company graphics and logos with excellent quality (relatively low compression). Whatever your use-case might be, our “Convert Image to GIF” API iteration makes this file transition straightforward with a single “input file” parameter.

#### Photoshop PSD Conversions

Adobe Photoshop is an extremely popular professional image editing platform, which means converting images to the platform’s native format, PSD, is a must-have service for many content creators. These files are notable for their immense pixel retention; they store an incredible amount of color and exhibit excellent quality relative to their size. The “Convert Image to PSD” API iteration will help you prepare images intended for Photoshop manipulation at scale; once your photo editing operations are complete, you can then convert the edited images back into another of our supported image formats just as easily.

#### BMP & TIFF Conversions

Original bitmap files like BMP and TIFF are decades old, yet they remain very popular among photographers due to their natural affinity for storing high-quality images. One of the chief advantages of using BMP is its ease of compatibility with Windows operating systems, whereas TIFF format offers considerably more flexibility (supporting lossy AND lossless images), dominating within the photo printing and publishing fields. Our “Convert Image to BMP” and “Convert Image to TIFF” APIs will make either format transition a seamless addition to your image processing operations.

#### WebP Conversions

While JPG and PNG format are exceedingly common image formats, they also have a quiet tendency to slow web pages down – at least, relative to WebP. This relatively lesser-known format (originating circa 2010) is the product of Google’s attempt to increase web page loading speeds with a new, lightweight file type. With WebP files, you can achieve an excellent degree of image quality at around 75% of the size of PNG and JPG formats. Most common browsers now support WebP natively, and Google Chrome will even encourage the conversion of files to WebP format. It’s a no-brainer conversion if you’re looking to speed up your web pages, and our “Convert Image to WebP” API makes it straightforward to accomplish that transition in a single step.  
  
  
  
[API Spotlight: Cloudmersive Barcode APIs](/blog/API-Spotlight%3a-Cloudmersive-Barcode-APIs)  
11/14/2022 - Brian O'Neill  
  
  


### API Spotlight: Cloudmersive Barcode APIs

![Barcode Scan](https://docs.cloudmersive.com/api/image/blog-image/980164dd-cb43-492d-b198-0891dc7a4b85) Thousands of global businesses - especially those involved in the retail, healthcare, and transportation sectors - need the help of consistent, reliable barcode services to operate at peak efficiency. All told, that means having the capacity to lookup, scan, and generate barcodes on command - and thankfully, there are Cloudmersive APIs available for all those needs. With access to our Barcode API endpoint, you can quickly generate a variety of the most common barcode types (including QR, UPC & EAN) as PNG files and scan/recognize images of many diverse barcode types with ease. In addition to those options, you can search & return information specifically for EAN barcode values (using the numerical barcode value only). Below, we’ll take a closer look at our barcode scanning and barcode generation services.

#### Barcode Generation

Our Barcode Generation APIs represent three of the most common barcode types used across the globe: QR, UPC, and EAN. While QR codes have gained widespread contemporary popularity largely in thanks to the proliferation of smartphones, the latter two options are older GS1 standards with decades of history in international commerce. UPC and EAN barcodes are very similar in practice, and the primary difference between them comes down to regional/geographical preference; UPC barcodes are used most commonly in the USA and Canada, while EAN barcodes are more widely seen throughout the rest of the world. Generating QR barcodes with our QR API is as simple as inputting any text string of your choosing; the API will return a 2D PNG file containing a unique QR barcode which can be scanned by any handheld photographic device. QR codes offer a fantastic way to make a variety of content - such as websites, menus, or notes of any kind - available for anyone with a smartphone camera. If it’s EAN or UPC barcodes you’re looking for, there’s a crucial distinction to be made within each category: does your product require EAN-13 or EAN-8; UPC-A or UPC-E? While EAN-13 and UPC-A barcodes are the “standard” barcode length for each category, the reality is that not all retail products are physically big enough to present these 13- & 12-digit barcode values (or the larger line patterns that represent them). For smaller products with less real estate to offer, EAN-8 and UPC-E offer a compact solution, containing only 8 and 6 digits respectively - and significantly compressed line patterns. All four of these common barcode types are readily available to you via their own individual Barcode API iterations, requiring nothing more than numerical barcode values to generate linear codes for your business at scale.

#### Barcode Scanning

If a huge number of businesses require the capacity to generate barcodes, exponentially more require the ability to scan them. Accepting inputs in common image formats (i.e., PNG, JPG, etc.), our Scan Barcode Image API will return clearly labeled strings identifying both the general barcode type and the specific barcode number represented within the input barcode image. A variety of barcode types are supported for this operation, including some of the most common (UPC, EAN, QR, etc.) and relatively less common (AZTEC, CODABAR, MSI, etc.) types out there. As a final note - for any EAN product values obtained through this Barcode Scanning operation, you can employ our EAN Lookup API to instantly return detailed product data.  
  
  
  
[API Spotlight: Natural Language Processing Analytics](/blog/API-Spotlight%3a-Natural-Language-Processing-Analytics)  
11/7/2022 - Brian O'Neill  
  
  
Have you ever wondered if there’s a way to quantify how customers collectively feel about your products based on online reviews? Alternatively, perhaps you’ve wondered if there’s a way to systematically protect all your content from profane language and hate speech? With the help of NLP Analytics, you can accomplish all the above, uncovering important customer insights, more easily moderating your online content, and much more. In this article, we’ll take a closer look at three of our five NLP Analytics APIs and how they can make a big difference for your business. ![silver laptop](https://docs.cloudmersive.com/api/image/blog-image/d36dc510-834d-4c54-a15a-9cf5918db73c)

#### Sentiment Analysis & Classification API

The purpose of Sentiment Analysis is to find out how a certain person – or group of people – feels about a given subject based on how they write about it. Are their feelings positive, negative, or neutral? With a Sentiment Analysis API in your toolbelt, you can gain a better understanding of how your customers feel about your products and make better informed marketing decisions as a result. Its use isn’t limited to only customer centric businesses, however – there are relevant applications of this technology in just about any professional sector you can think of. To quantify written feelings, this advanced API rigorously analyzes language in a body of text and provides a Sentiment Classification Score value between -1.0 and 1.0. Scores closer to -1.0 indicate a negative view on a given subject, while values closer to 1.0 indicate a positive view; scores closest to zero instead suggest a more neutral sentiment. This API additionally returns a Sentiment Classification Result string based on the score provided; the possible responses are positive, negative, or neutral.

#### Profanity & Obscenity Detection API

Bad language is a bad representation of your brand – even if it comes from voices outside your organization. When customers come away feeling disappointed, aggrieved, or even elated from an experience with any of your products, they can occasionally express themselves with inappropriate language, which – especially in the case of online reviews – can create an unsavory environment for new patrons. This API performs a practical content moderation service by employing advanced analysis to detect profane/obscene language from text; it returns a Profanity Score Result between 0.0 – 1.0, with values closer to 0.0 suggesting very little profanity, and values closer to 1.0 indicating a high degree of profanity.

#### Hate Speech Detection API

It goes without saying that content containing Hate Speech should have no place anywhere on any company’s platform. Considering the overwhelming importance of flagging and removing Hate Speech content – paired with the difficulty human content moderation teams face in scouring insurmountable written materials online – Hate Speech Detection can make a huge difference, reducing the burden of manual review through automatic detection of discriminatory language in various forms. Much like the previously described Profanity Detection API, this API provides a Hate Speech Score Result between 0.0 and 1.0, with values on the lower end of the spectrum indicating very little (or no) hate speech was detected, and scores on the higher end of the spectrum suggesting Hate Speech was, in fact, present. Values above 0.8 indicate a particularly high likelihood that Hate Speech was rampant in a body of text, making it easy for content moderation groups to efficiently triage their response.  
  
  
  
[API Spotlight: Virus Scanning APIs](/blog/API-Spotlight%3a-Virus-Scanning-APIs)  
8/5/2022 - Brian O'Neill  
  
  


### **CLOUDMERSIVE VIRUS SCANNING APIs**

![vertical view of typing](https://docs.cloudmersive.com/api/image/blog-image/9b3eac3f-6210-4f3d-96fd-83f6dc2a05b8) When it comes to protecting your data in-storage against viruses, malware, and other dangerous files, Cloudmersive Storage Protect offers comprehensive & customizable solutions to fit your business’ needs. But what does the heavy lifting behind the scenes? Your answer: the Cloudmersive Virus Scanning APIs. In this article, we’re going to shift our gaze away from the Storage Protect product and train it on those underlying Virus Scanning APIs instead. They can be used independently to scan individual files and websites for malicious content, each benefitting from constant cloud-based updates to stay ahead of the curve. With fast (sub-second) response times, you can ensure you’re instantly capable of uncovering threats hidden within a suspicious file or link.

### **SCAN A FILE FOR VIRUSES**

The first iteration of the Virus Scanning API offers a simple and effective method for scanning individual files for viruses. To that end, this API leverages more than 17 million virus and malware signatures – and that list grows constantly with continuous cloud-based updates to bolster its policies against evolving threats. Files containing viruses, malware, trojans, ransomware & spyware (among others) can be identified immediately and quarantined or deleted in subsequent operations (for example, you may use our Zip, Encrypt & Quarantine API to automatically create an encrypted zip file composed of the infected file or files). The API response will first show a Boolean indicating if there was a clean result, and if False, it will display the name of the infected file along with the name/type of virus identified within it.

### **ADVANCED SCAN A FILE FOR VIRUSES**

Building on the basic file scanning iteration, the Advanced Scan version simply expands security coverage to identify a wider variety of threats. That means you’re still covered when it comes to viruses, malware & the like, and you’re now well-positioned to deal with invalid files, executable files, scripts, and much more. This iteration allows you to customize (via Booleans) whether various threats such as Insecure Deserialization strings, Macros, XML External Entities, and Password Protected Files should be blocked outright (the default & recommended option is to block all the above). The API response will include Booleans indicating which content threats were present. In addition, it will provide strings indicating the file name of any files containing viruses & the type of virus which was detected. This API iteration will also provide basic content information about whether scanned files contained JSON or XML, and if any of the included file contained an image within them.

### **SCAN A WEBSITE FOR MALICIOUS CONTENT & THREATS**

Website URLs represent a slightly different threat potential than objects/files and serialized strings. The website threat detection iteration of the Virus Scanning API can uncover common threats hidden within a website URL, such as phishing attempts, which seek to mimic landing pages and login pages of legitimate websites to fool unsuspecting people. This iteration will also identify any viruses found in files within the input website. If a threat is identified, the API response will first identify which type of threat was represented, before divulging the name of any infected files & the viruses those files contained. Social engineering scams like phishing are designed to exploit our implicit trust of URLs that look and feel the way they should, and most often were sent from sources we typically trust – so it’s important to implement comprehensive, “no-stone-unturned” policies against them.  
  
  
  
[API Spotlight: Image Optical Character Recognition APIs](/blog/API-Spotlight%3a-Image-Optical-Character-Recognition-APIs)  
7/26/2022 -   
  
  
![filling out document](https://docs.cloudmersive.com/api/image/blog-image/a3a85cd2-9c99-438b-845b-05d6f86fdace)

### **Image OCR Overview**

This week, we’re putting a spotlight on our Image Optical Character Recognition (Image OCR) APIs. These APIs make it possible to extract text directly out of scanned images and photographs of documents, and several are designed with specific use-cases like Receipt expensing, Business Card information extraction, and Business Form processing. All are equipped to recognize dozens of common international languages, with English set as the default unless specified otherwise. In addition, all employ an advanced ‘recognition mode’ option by default which increases the fault tolerance for skewed document images, consuming around 28-30 API calls to ensure the highest possible end-product. Let’s take a closer look at how our Image OCR APIs can impact your business.

### **Converting Scanned Documents into Text**

There are three iterations of our OCR API designed specifically to recognize images of documents which have been scanned rather than photographed (i.e., with a phone or other handheld device). These iterations include:

  * **Convert Scanned Images to Text**
    * This iteration executes a vanilla OCR operation on scanned images, returning a text result with a “MeanConfidenceLevel” score indicating the degree to which the API believes the operation was successful.
  * **Convert Scanned Images to Words/Text with Location**
    * This iteration returns scanned image text with metadata about the words within the image. This includes a “WordText” string, line & word numbers, x/y coordinate locations, width, height, and more.
  * **Convert Scanned Images to Lines/Text with Location**
    * This iteration returns scanned image text with metadata about specific lines within the document. This includes information about each word within the specified line, including a “WordText” string and other metadata also returned by the Words/Text API iteration.



### **Converting Photographs of Documents into Text**

These iterations of the Image OCR API are intended only for use on images captured by a smartphone or other handheld convenience-based devices. They automatically attempt to compensate for problems commonly found in handheld photos of documents, such as crooked backgrounds and uncertain lighting. These iterations include:

  * **Convert a Photo of a Document into Text**
    * This iteration will return text from the input photo along with a “MeanConfidenceLevel” score indicating the degree to which the API feels it was successful.
  * **Convert a Photo of a Document OR Receipt into Words with Location**
    * This iteration returns photos of generic documents or receipts with the location of each text element. It can be used effectively on receipts, though not with the same degree of accuracy as the Receipt OCR API. A diagnostics mode can be enabled for this API by entering a ‘true’ Boolean in the relevant parameter; this is set to ‘False’ by default to streamline performance.



### **Converting Receipts, Business Cards, Generic Forms & Stored Template Forms into Structured Text**

These last iterations of the Image OCR API are excellent at digitizing text from photographs (NOT scanned images) of documents with specific structures. Such documents include Receipts (most often used for expensing employee transactions), Business Cards (most often containing useful information for generating sales leads), and Business Forms (may include basic new-employee handouts invoices, and more). These iterations include:

  * **Recognize a Photo of a Receipt; Extract Key Business Information**
    * Along with the receipt total & subtotal, this iteration will extract key information from a receipt including the name, website, physical address & phone number of the business in question, as well as a complete list of items on the receipt and their respective prices.
  * **Recognize a Photo of a Business Card; Extract Key Business Information**
    * This iteration will extract key information from an input photo of a business card, including the name & title of the person on the card, the name of their business & their business address, and any phone number and email that is available on the card.
  * **Recognize a Photo of a Form; Extract Key Fields & Business Information**
    * This iteration allows you to pull specific data from a photo of a Business Form by customizing the fields of data to be extracted. There are a wide variety of options available with this iteration; to view the entire response model, refer to this iteration’s documentation dropdown on the Cloudmersive API Console.
  * **Recognize a Photo of a Form, Extract Key Fields using Stored Templates**
    * This iteration allows you to extract information from a form using stored templates. These templates can be configured in a configuration bucket by logging into the Cloudmersive Management Portal and navigating settings > API Configuration > Create Bucket. The bucket ID and Secret Key are required to execute OCR based on the stored template.

  
  
  
  
[API Spotlight: Speech Recognition (Speech-To-Text Conversion)](/blog/API-Spotlight%3a-Speech-Recognition-\(Speech-To-Text-Conversion\))  
7/22/2022 - Brian O'Neill  
  
  
![Mic in front of camera](https://docs.cloudmersive.com/api/image/blog-image/711bdd55-683e-4263-a36a-8b9c5ac49911)

### **Speech Recognition Overview**

Speech recognition technologies have quietly proliferated throughout the consumer hardware & software technology landscape over the last decade. In a consumer context, it’s easy to think of speech recognition as little more than a useful gimmick, resigned to helping us find out the day’s weather or record our shopping lists on the fly. This consumer application of speech recognition is only the tip of the iceberg, however. Speech recognition has exciting commercial applications, offering enterprises the ability to transcribe audio more efficiently into plain text and derive meaningful insights from it. The Cloudmersive Speech Recognition API leverages deep-learning artificial intelligence to transcribe audio files into plain text, and it supports either MP3 or WAV file input. With the Cloudmersive Speech Recognition API, speech-to-text can begin to play a supplementary role in your business’ content moderation efforts. Further, it can transform the way your business extracts value from lectures, interviews, and other industry events where spoken language dominates the playing field, and it can even increase the access that physically impaired website visitors or employees have to your online resources.

### **Speech Recognition for Content Moderation**

Nearly all forms of externally sourced media pose some risk to your system, whether from a security or NSFW (not safe for work) point of view. Image and video files, for example, are not just at risk of containing viruses and malware – they can contain pornography or racy material as well, which can leave your business in a sticky legal situation. Valid audio files can similarly hide NSFW policy breaches, and that content can go undetected if an effective audio content moderation solution is not introduced. Speech recognition technology plays a pivotal role in moderating audio content, providing the means to transcribe audio into plain text for subsequent NSFW text analysis. By incorporating a combination of Cloudmersive Speech Recognition and NLP (Natural Language Processing) APIs, your business can rapidly uncover profanity, hate speech and other forms of inappropriate language or harassment hiding within the waves of an audio file.

### **Speech Recognition for Speech-To-Text Analysis**

Content moderation isn’t the only reason to convert speech to text. Audio files can arrive in your database as raw recordings of lectures, speeches, interviews, or other impromptu oratory from relevant industry events. Speech-to-text deeply enhances the value of such audio, first by cutting out the need to take attention-diverting notes at the event itself (or waste precious time manually transcribing audio recordings later). The automatically transcribed text from such audio files not only creates a better form of storage redundancy & search optimization for its recorded contents, but it also opens a door to analyze the extracted text more efficiently and derive deeper insights from it. For example, text can be processed through your Cloudmersive Sentiment Analysis API to classify if the speaker had a positive, negative, or neutral feeling towards a given subject. In the same way, it can undergo Subjectivity Analysis to help identify the degree to which the speaker might’ve been subjective or objective regarding their topic. If two lecturers spoke on the same subject, Semantic Similarity Analysis can be applied to determine the degree to which they might’ve meant the same or different things based on the phrasing they employed. Deploying our Speech APIs in conjunction with your Cloudmersive NLP APIs can transform the value of your recorded audio and establish the value of its content.

### **Speech Recognition for Physically Impaired Users**

In the context of both consumer and commercial technology, Speech Recognition can be leveraged to assist people who are managing short or long-term physical impairments. By providing a means of transforming dictated audio into text, your business can empower those who are physically unable to type effectively on a keyboard and expand their access to important resources. This application of Speech Recognition can help transform the way your business interacts with its users and employees alike.  
  
  
  
[Announcing Storage Protect for Azure Blob](/blog/Announcing-Storage-Protect-for-Azure-Blob)  
6/14/2022 - Brian O'Neill  
  
  


### **ANNOUNCING STORAGE PROTECT FOR AZURE BLOB**

![Tech office](https://docs.cloudmersive.com/api/image/blog-image/18930af1-a5d3-40c0-b904-1fd852768307) We’re excited to announce the newest release of our Storage Protect Cloud Security solution for Azure Blob. This product now has 4 unique iterations for protecting Azure Blob, Google Cloud Storage, AWS S3 and SharePoint Online Site Drive instances. Threats to your data storage can include a wide variety of viruses and malware, scripts, executables (and much more), and they can even include breaches of NSFW policy in image files. Storage Protect provides dynamic layers of protection against a huge number of potential threats to your data storage, and it leverages two powerful Cloudmersive APIs to get the job done.

### **STORAGE PROTECT: VIRUS SCANNING FOR AZURE BLOB**

Every cloud storage product, including Azure Blob, contains built-in security policies designed to protect your data in basic ways. These policies aren’t enough to stay on top of the ever-evolving threat landscape, however; that’s why our Virus Scanning API uses a growing list of 17 million+ virus and malware signatures to always keep your threat profile ahead of the curve. Security policy customization is key to ensure our product fits your organization's needs. To that end, Storage Protect allows you to configure one of two iterations of the Cloudmersive Virus Scanning API to dictate your own policies. These iterations are displayed as Basic and Advanced options when configuring your product. With the Basic API capabilities, you’ll enable multi-threat scanning across viruses, malware, trojans, ransomware, and spyware, with a wide variety of file formats supported. The Advanced option builds on the Basic option to block additional threats beyond the scope of viruses and malware, such as scripts, executables, invalid files, and more. Each Virus Scanning API iteration boasts a sub-second response time, so you’ll get the results of your scan almost instantaneously. To help you manage scenarios where threats are identified, Storage Protect provides simple, customizable outcome actions allowing you to delete or quarantine the infected files outright. Clean files can be just as easily tagged as “Clean”, so you’ll always end up with an easy-to-navigate scan result no matter what.

### **STORAGE PROTECT: AI CONTENT MODERATION FOR AZURE BLOB**

Protecting your data isn’t always about scanning for viruses and malware. Image files stored in your database can quietly contain breaches of NSFW (Not Safe for Work) policies, such as pornographic or racy material, and unsolicited viewership of such images can sometimes result in sexual harassment litigation. Storage Protect now offers AI Content Moderation services specifically geared towards identifying racy or pornographic material within images by automatically reviewing each file's respective pixels. This task is performed by the underlying NSFW Classification API, which provides a score between 0.0 – 1.0 for each image file based on the degree to which the contents of the file consist of racy or pornographic content. Files with zero racy content typically get scores between 0.0 – 0.2 and receive an accompanying Classification Result string which appropriately describes the score range as “Low Probability.” Scores between 0.2 – 0.8 have an increasing likelihood to contain some degree of racy content and receive a “Medium Probability” string. Any scores above 0.8 are considered “High Probability” to contain illicit content; scores closest to 1.0 are the most likely to contain outright pornography. This AI Content Moderation feature can be enabled by electing the Advanced Scan option in Storage Protect. Using the dynamic scoring system provided through the NSFW Classification API, you can easily tailor NSFW content scans to identify images based on your company’s specific NSFW policies. With this feature configured, your business can save a lot of time, energy and resources that might’ve otherwise spent manually reviewing image uploads to achieve the same classification result.

### **LOOKING AHEAD**

When we add new features to Storage Protect, you’ll be the first to know about it. We’ll keep you posted as our flagship security product continues to evolve in new and exciting directions.  
  
  
  
[Announcing Storage Protect for Google Cloud](/blog/Announcing-Storage-Protect-for-Google-Cloud)  
5/24/2022 - Brian O'Neill  
  
  


### **STORAGE PROTECT FOR GOOGLE CLOUD**

Data is the most precious commodity for any online business. However, that data can sometimes work against us, containing hidden threats and breaches of policy. What if a few new files in your Google Cloud storage instance are hiding ‘dormant’ viruses or malware? What if a website user uploaded a product review photo that contains sexually explicit content, violating your company’s NSFW (not safe for work) policy? These deceptive, threat-bearing files create big problems when they’re mistakenly trusted and opened. It’s easy to assume that internally stored files are always safe by right of existing inside a professional database, and not outside of it. For that reason, it’s best to check your data for hidden threats in storage. Thankfully, with Cloudmersive Storage Protect for Google Cloud, you can easily scan new files as they enter Google Cloud storage to find out if any viruses or malware lay hidden within them. Further, you can configure an AI Content Moderation feature which scans image files and determines if they contain NSFW (Not Safe for Work) image content of a racy or pornographic nature.

### **VIRUS SCANNING API**

Equipped with over 17 million virus and malware signatures and bolstered with regular cloud-based updates, the underlying Cloudmersive Virus Scanning API in Storage Protect checks underneath the ‘hood’ of each new file in Google Cloud storage and finds out if any such file contains a threat. This flagship content security feature can be configured to perform either Basic or Advanced scans, with the former covering primarily virus and malware detection, and the latter expanding upon that coverage to include 360-degree content protection across executables, invalid files, scripts, and restrictions on accepted file types.  
After each scan, problematic files can be assigned special outcome actions, including the option to quarantine or delete those files outright. Clean files can be easily filtered and tagged as well, ensuring you have full control on either end of the spectrum.

### **A.I. CONTENT MODERATION API**

The Cloudmersive NSFW Image Classification API is deployed alongside the Cloudmersive Virus Scanning API in Storage Protect, leveraging Machine Learning to ensure image files in storage are safe and appropriate for everyone to view (based on each customer’s unique NSFW policy). This NSFW Classification feature can be configured by enabling the Advanced Scan feature of Storage Protect. With NSFW scanning option configured, new image files entering cloud storage will be analyzed and categorized based on the degree of racy or pornographic content contained within them. To quantify such content, a “Profanity Score Result” is automatically provided for each scanned image, with values ranging from 0.0 to 1.0. Scores on the bottom end of that range indicate a lower likelihood that the image contains racy or pornographic content, while higher numbers indicate that such content is likely to exist in the image. Accompanying the Profanity Score Result value, a natural language description is provided indicating whether the image has a Low (0.0 – 0.2), Medium (0.2 – 0.8) or High (0.8 – 1.0) probability to contain NSFW content. Storage Protect is also currently available for AWS S3, Azure Blob & SharePoint Online Site Drive storage instances.  
  
  
  
[New AI Content Moderation Feature Added to Storage Protect](/blog/New-AI-Content-Moderation-Feature-Added-to-Storage-Protect)  
5/6/2022 - Brian O'Neill  
  
  
![AI Content Moderation for Storage Protect](https://docs.cloudmersive.com/api/image/blog-image/b5b2cbf1-8e45-45ba-a7ca-d7ca28675cf1) We are excited to announce the addition of a brand-new AI Content Moderation feature to Storage Protect. This new feature uses Machine Learning to automatically review and classify image files uploaded to cloud storage based on the degree to which they may be NSFW (Not Safe for Work). The type of Not Safe for Work (NSFW) content being screened for by this new addition is strictly that of a racy or pornographic nature.

### Storage Protect Overview

Storage Protect is Cloudmersive’s leading edge in-storage cloud security product, designed to accompany Amazon Web Services (AWS) S3, Azure Blob, Google Cloud (GCP) and SharePoint Online Site Drive storage.

### Storage Protect: Virus Scanning Service

Under the hood, the baseline Storage Protect product calls upon the Cloudmersive Virus Scanning API to scan new files as they enter a customer’s cloud storage instance. The Virus Scanning API is equipped with over 17 million virus and malware signatures and boasts a sub-second typical response time due to high-speed, in-memory scanning. Customers can configure Storage Protect to perform either Basic or Advanced scans. The former checks files for viruses, malware, trojans, ransomware & spyware, and the latter expands that list to include executables, invalid files, scripts, and more. Files are classified as “clean” or “infected’ post-scan and any identified threats can then be quarantined, deleted, or otherwise dealt with based on outcome actions predetermined by the customer.

### Storage Protect: AI Content Moderation Service

Storage Protect now includes the services of the Cloudmersive NSFW Image Classification API. This additional API uses Machine Learning to scour the pixels within each newly uploaded image file and classify the degree to which that content is racy or pornographic (or neither). This operation returns both a classification score (scaled between 0 and 1, with lower values indicating very little racy content was detected) and a probabilistic description of the degree of raciness in the image based on that score (0 – 0.2 is low probability; 0.2 – 0.8 is medium probability; 0.8 – 1.0 is high probability) for each file. For any additional information, review our whitepaper on AI Content Moderation for Storage Protect, or contact your Cloudmersive representative more details.  
  
  
  
[JSON Insecure Deserialization API Spotlight](/blog/JSON-Insecure-Deserialization-API-Spotlight)  
5/4/2022 - Brian O'Neill  
  
  
![Content Security APIs](https://docs.cloudmersive.com/api/image/blog-image/45b839ac-e05f-4c24-a26a-d090584861dd)

### What is JSON Insecure Deserialization?

Better known as a “JID Attack” in short form, JSON Insecure Deserialization is a form of cyber-attack in which data, controlled by a malicious user, is deserialized by a website. Serialization refers to the course of turning complex data structures into a format that can be exchanged as a stream of sequential bytes, and Deserialization refers to the process of turning that byte stream back into a replicated version of the original object (same state as when the original data was serialized) so that the website can interact with it. When this serialization process is manipulated successfully by an attacker, a major vulnerability has been exposed. The attacker can now sneak harmful data into your application’s code, causing all kinds of problems.

### JSON Insecure Deserialization Detection API

If there’s a JSON Insecure Deserialization attempt hiding in suspicious text input, the JID Detection API will know all about it. When that text input is provided, the JID Detection API responds in seconds by determining first if the operation was successful (providing a true or false value). If true, the following response will indicate whether the text input contained a JID attack (again providing a true or false value).

### Cloudmersive Content Threat Detection

It doesn’t stop with just JSON Insecure Deserialization attacks – there are several other forms of text threats that can cause immense harm to your applications. Cloudmersive has you covered with APIs detecting SQL Injection Attacks, Cross-Site Scripting Attacks and XML External Entity Attacks.  
  
  
  
[API Spotlight: Simple Barcode Generation for your Business](/blog/API-Spotlight%3a-Simple-Barcode-Generation-for-your-Business)  
8/19/2021 - Laura Bouchard  
  
  
![API Spotlight - Barcode APIs](https://docs.cloudmersive.com/api/image/blog-image/5a1e3a61-6cfd-4596-ad67-490c18016007) Barcodes are used by businesses all over the world to aid in tracking, purchasing, inventory, and more. By leveraging these codes, organizations can keep accurate records of their products, supplies and inventory that are essential to operations. This week we will be shining our API spotlight on a few of our Barcode APIs that can assist in improving the overall experience from both a user and business standpoint.

### UPC-A Barcode API 

The first function we will discuss is the full email address validation API. Email is one of the most universal communication tools across the world, which also means it is one of the most popular targets for attacks. This API allows you to check for syntactic correctness, identify the mail server, and contact the server to validate the existence of the account without sending any emails.

### EAN-13 Barcode API 

Next up is our full URL validation API; similar to the previous function, this validation API will perform a full, in-depth validation of its target input, which in this case is a URL address. By checking if the input URL is syntactically valid, whether it exists, and whether the endpoints pass virus scan checks, you can ensure each URL you interact with is free from threats.

### QR Barcode API

Our last featured API is the IP Intelligence API. This is a multi-use function that will not only scan a single IP address against a database of known threat IPs, bots, and Tor exit nodes, but will also provide the physical location of the address. We will be sharing API spotlights and new features over the coming weeks, so be sure to check back in with us soon.  
  
  
  
[New Convert API to Transform your DOCX Tables](/blog/New-Convert-API-to-Transform-your-DOCX-Tables)  
8/12/2021 - Laura Bouchard  
  
  
![Convert Fill DOCX Table API](https://docs.cloudmersive.com/api/image/blog-image/8efe8039-05e4-4b0a-8f34-d35a8285013f) One of our company’s most important goals is to create new and innovative solutions to make your job easier. We know how difficult it can be to edit an online document, so we have several APIs available in our library that can assist you with the undertaking. Today, we are pleased to share that we have added a new Convert API to our offerings that can be used to transform your Word DOCX files by instantly filling in tables with data.

### Obtain an Editing URL

Before you can use this new API, you will need to open up the process and obtain an editing URL by using the Begin Editing API function, which can be found in the Convert API library. To ensure the security of your information, the URL is stored in-memory cache and expires after 30 minutes.

### Filling the Table with Data

Once you have the editing URL, you are ready to call the Table Fill In function. This easy process will allow you to replace placeholder rows in a table in a DOCX file using one or more templates. For the input request, you will need the following information: File URL, File Data, Table Start Tag, Table End Tag, Data to Fill in, Cells, Target Tag (string), and Replacement Value (string).

### Downloading the Result

Once you have completed the editing of your Word document, you can use the Finish Editing API function to close out processing and download the updated result. Each of these APIs, including the new, featured API, are available for testing and use in a variety of programming languages in our API Console.  
  
  
  
[Now Introducing: The Security Threat Detection API](/blog/Now-Introducing%3a-The-Security-Threat-Detection-API)  
7/29/2021 - Laura Bouchard  
  
  
![Security Threat Detection](https://docs.cloudmersive.com/api/image/blog-image/1096f291-8294-497d-9f6c-7fbcd8a32fd1) The threats that we encounter in cyber space are constantly growing and changing; it seems that as soon as we protect ourselves from one type of attack, another new or customized attack emerges. Here at Cloudmersive, we are dedicated to providing you with high-quality and up-to-date security coverage to keep you ahead of the curve in the current, dynamic environment. As part of this commitment, we are excited to announce the launch of our new Security Threat Detection API and Insecure Deserialization JSON API. Both APIs are detailed below and are available in the newly added Security API section of our API Console, which features several of our currently existing security APIs as well.

### Security Threat Detection API

This new API takes several types of threat detection and combines them into one, convenient package. Using Cloudmersive Intelligent Threat Detection, you can automatically scan any input string for a wide range of threats including Cross-Site Scripting (XSS), SQL Injection (SQLI), XML External Entities (XXE), Server-Side Request Forgeries (SSRF), and JSON Insecure Deserialization (JID).

### JSON Insecure Deserialization (JID) API

JSON Insecure deserialization is yet another type of cyber attack that is becoming prominent across the web. The threat involves the passing of manipulated objects through a web application that eventually results in control of the application. If the attack is successful, it can lead to denial of service, malicious code execution, and more. To protect against this vulnerability, the JID API will instantly detect these type of attacks from text input. For more information on these and our other security APIs, check out our [Security Threat Detection API](https://cloudmersive.com/security-threat-detection-api) page.  
  
  
  
[XXE Threat Detection Now Available in Advanced Virus Scan API](/blog/XXE-Threat-Detection-Now-Available-in-Advanced-Virus-Scan-API)  
7/15/2021 - Laura Bouchard  
  
  
![XML External Entity Threat Detection API](https://docs.cloudmersive.com/api/image/blog-image/f7de9e09-9540-4314-92c5-c3007cb10adf) We are excited to announce that we have added an XXE threat detection feature to our Advanced Virus Scan API! XML External Entity (XXE) threats are targeted attacks that exploit vulnerabilities within Document Type Definitions (DTD) in XML parsers to replace entities and cause a denial of service; they can also utilize Server-Side Request Forgery (SSRF) to gain access to sensitive data. Since these threats fall outside the protection of your basic anti-virus software, the job of protecting a web application from these attacks will often fall to the developer – this is where our new capability steps in.

### Advanced Virus Scanning API

Our Advanced Virus Scan API provides 360-degree content security with customizable parameters to meet your needs. The AllowXMLExternalEntities feature joins several other options such as AllowMacros, AllowExecutables, and AllowInvalidFiles to assist in fully protecting you and your customers.

### How It Works

This capability is very easy to set up; simply set the parameter to false to block XML External Entities, other threats embedded in XML files, and other files that contain embedded content threats. If you wish to allow these file types, you can set the parameter to true, but the default setting is false (recommended).  
  
  
  
[Additional Features Added to Cloud Storage Protect](/blog/Additional-Features-Added-to-Cloud-Storage-Protect)  
7/8/2021 - Laura Bouchard  
  
  
![Cloud Storage Virus Scanning](https://docs.cloudmersive.com/api/image/blog-image/462f185a-15c4-46da-8130-dad30db88db1) Just a few months ago, we released our Cloud Storage Protect service and shared how you can integrate it into your AWS S3 and Azure Blob cloud storage. In case you missed it, this code-free service allows you to protect your content from viruses, malware, and other threats by automatically scanning uploaded files and objects according to security policies that you set up within your cloud storage containers. Now, we are excited to share that new features have been added which will enable you to take your virus scanning capabilities to the next level; details on the features are provided below.

### Advanced Scanning Options

Cloud Storage Protect now offers the same security policies that are available in our Advanced Virus Scan API; this includes the option to allow/block executables, scripts, macros, invalid files, password-protected files, and more.

### Block Download

Our next enhancement provides the option to block a download and/or move a file while scanning, enhancing the overall convenience and functionality of the service.

### Learn More

If you want to learn more about Cloud Storage Protect and see it in action, we have video tutorials for setting up and using the service in both AWS S3 and Azure Blob. [How to Scan AWS S3 File Uploads for Viruses](https://youtu.be/rdKPnqDxIuY) [How to Scan an Azure Blob Files for Viruses](https://cloudmersive.com/videos/How-to-Scan-an-Azure-Blob-for-Viruses)  
  
  
  
[Check out our New Document Conversion APIs ](/blog/Check-out-our-New-Document-Conversion-APIs-)  
7/1/2021 - Laura Bouchard  
  
  
![Cloud Document Conversion API](https://docs.cloudmersive.com/api/image/blog-image/abecd18a-9a2f-4f83-9d4f-9e62b6137b6f) Here at Cloudmersive, we know how important it is to establish a professional and engaging platform. This means that we are always looking for more possibilities to automate and streamline processes and improve the overall experience for developers and users. To assist with this goal, we would like to introduce seven new document conversion APIs that will optimize the processing of HTML documents.

### Append an HTML Tag to the HEAD Section

This function will allow you to add an HTML tag to the HEAD section of an HTML document.

### Get the Language Code

With this, you can quickly retrieve the language code of an HTML document, such as ‘en’ (English) and ‘de’ (German).

### Set the Language Code

Parallel to the above function, this function will allow you to set the language code of an HTML document; hundreds of country codes supported.

### Get the Rel Canonical URL

This function provides a quick way to retrieve the rel canonical URL, or preferred URL, from an HTML document.

### Set the Rel Canonical URL

In order to avoid duplicate content issues that can arise when the same content can be accessed from multiple URLs, this API allows you to set a rel canonical URL for an HTML document.

### Get the Sitemap URL

This function will retrieve the sitemap URL from an HTML document which will provide you with information on each URL, such as when it was last updated and how relevant it is in relation to other site URLs.

### Set the Sitemap URL

Our last function allows you to set the sitemap URL of an HTML document; this will assist search engines in locating all related URLs. All of these APIs are available for use in our wide selection of programming languages and can be found in the Cloudmersive API Console.  
  
  
  
[New API Offerings Added to Cloudmersive Mendix Connector ](/blog/New-API-Offerings-Added-to-Cloudmersive-Mendix-Connector-)  
6/24/2021 - Laura Bouchard  
  
  
In January, we shared details on our partnership with Mendix, a low-code platform that rapidly creates applications to serve business needs. Our offerings in the Cloudmersive Mendix Connector started out with some of our most popular APIs (virus scan, image recognition/processing, document conversion), and now we are pleased to announce that we have added a few new offerings to the portfolio! These new features are detailed below and can be accessed via the Virus Scan section of the connector.

### Zip/Unzip Files

Zip and Unzip options have been added that will allow you to create a Zip file from uploaded documents or unzip an unencrypted Zip file with just the click of a button.

### Encrypt/Decrypt ZIP Files

Within the Zip option, there is an ‘Encrypted Zip’ box you can check, which will prompt a password box to appear. Once your Zip file is created, you will be required to enter the password for access. On the flip side, we have also added a ‘Unzip Encrypted Zip’ option to the dashboard, which will allow you to automatically decrypt an encrypted Zip file.

### URL Screenshot

Our last addition is a URL Screenshot feature that allows you to screenshot a URL prior to clicking on it; this will provide an additional layer of security against potentially harmful links.

### See for Yourself:

If you would like a demonstration of these features, check out the short video we just released on our YouTube channel that will walk you through each of them: [How to Zip and Encrypt a File or Screenshot a URL in Mendix](https://cloudmersive.com/videos/How-to-Zip-and-Encrypt-a-File-or-Screenshot-a-URL-in-Mendix)  
  
  
  
[Virus Scanning Reverse Proxy Server Now Available for Debian 10 Linux](/blog/Virus-Scanning-Reverse-Proxy-Server-Now-Available-for-Debian-10-Linux)  
6/17/2021 - Laura Bouchard  
  
  
We are excited to announce that our Virus Scanning Reverse Proxy Server is now available for Debian 10 Linux! This installation option is in addition to the Windows 2016 server and will improve the accessibility of the Reverse Proxy Server for more users. In case you need a reminder, our Reverse Proxy service is a no-code solution that can be integrated into your web application to automatically scan files for viruses before they reach your server. Now, we will take a quick look at what you need for the Debian 10 Linux installation.

### Installation Pre-requisites

Debian 10 with a graphical user interface (GUI) enabled is required, and you should be sure to run the latest software updates prior to initiating the installation.

### Hardware Provisioning

You will need to provision the hardware or virtual machines and the associated infrastructure. For single-node operation, a single server is required; for high-availability, a minimum of two nodes is required.

### Performing the Installation

To perform the installation, simply navigate to the Cloudmersive Management portal, log in, and click on Private Cloud Deployment. To install a node in the cluster, you will select the corresponding node, then click on Debian 10 Linux and follow the instructions for your environment.

### Outbound Network Connectivity

Some outbound network connectivity from your Linux Server to Cloudmersive services is required; refer to the instructions to whitelist or allow the outbound network connectivity where appropriate.  
  
  
  
[New Export Feature Added to API Analytics ](/blog/New-Export-Feature-Added-to-API-Analytics--)  
6/10/2021 - Laura Bouchard  
  
  
Our API Analytics function, which allows you to monitor API usage for each of your API keys via audit log, just got even better with the addition of an export feature. This new feature will export your usage data to an Excel document, which can provide organized and valuable insight on which APIs you or your business are utilizing the most.

### Step 1 – Access API Analytics

To access API Analytics, simply head to your Cloudmersive Management Portal and select the API Analytics function.

### Step 2 – View & Export Logs

Once you reach the API Analytics page, you will encounter a few drop-down menus. Here, you will select the API key, API type (i.e. Document and Conversion API, OCR API, etc.), and the maximum number of results you wish to view. Now, before you click the ‘View Logs’ button, you can check the ‘Enable Export’ box to generate an Excel report of the results.

### Step 3 – Download Results

At the top of the Data section, there will be a Download Excel link with an arrow next to it; to download your results, simply click on the link and open the newly generated Excel file. That’s it for this week, but be sure to check back in with us soon as we share more featured APIs and new services in the coming weeks.  
  
  
  
[API Spotlight: Validation APIs to Protect Your Data ](/blog/API-Spotlight%3a-Validation-APIs-to-Protect-Your-Data-)  
6/3/2021 - Laura Bouchard  
  
  
Last week we highlighted a couple of our conversion APIs, so this week we will be switching things around and shining the spotlight on a few of our validation APIs. The following APIs can be used alone or in tandem to protect your personal or business data from various cyber threats.

### Full Email Address Validation API

The first function we will discuss is the full email address validation API. Email is one of the most universal communication tools across the world, which also means it is one of the most popular targets for attacks. This API allows you to check for syntactic correctness, identify the mail server, and contact the server to validate the existence of the account without sending any emails.

### Full URL Validation API

Next up is our full URL validation API; similar to the previous function, this validation API will perform a full, in-depth validation of its target input, which in this case is a URL address. By checking if the input URL is syntactically valid, whether it exists, and whether the endpoints pass virus scan checks, you can ensure each URL you interact with is free from threats.

### IP Intelligence Validation API

Our last featured API is the IP Intelligence API. This is a multi-use function that will not only scan a single IP address against a database of known threat IPs, bots, and Tor exit nodes, but will also provide the physical location of the address. We will be sharing more API spotlights and new features over the coming weeks, so be sure to check back in with us soon.  
  
  
  
[API Spotlight: Edit PDFs to Fit Your Needs](/blog/API-Spotlight%3a-Edit-PDFs-to-Fit-Your-Needs)  
5/27/2021 - Laura Bouchard  
  
  
When you think of conversion APIs, you generally think of converting a document from one format to another, such as CSV to XLSX or DOCX to PDF. However, in today’s blog post, I’d like to shine the spotlight on a couple of our popular PDF conversion APIs that allow you to edit a PDF document to (literally) fit your needs.

### Reduce PDF File Size API

Our first featured API reduces PDF file size by digitally compressing and optimizing its contents. This can be particularly helpful when sharing large documents such as contracts or manuals via email.

### Resize PDF API

While the previous API focuses on changing the file size of a PDF, this API focuses on altering the actual paper size. This function will allow you to instantly adjust the paper size of a PDF document to meet the requirements of any project. The paper sizes run from A7 (the smallest) to A0 (the largest); for reference, the standard letter size is A4. As a reminder, these conversion APIs can be used in over a dozen programming languages, as well as no-code/low-code connectors such as Power Automate and Logic Apps. To test them out, simply head over to the Convert API section of our API Console.  
  
  
  
[Feel the Flow with our Power Automate Connector How-to Videos](/blog/Feel-the-Flow-with-our-Power-Automate-Connector-How-to-Videos)  
5/12/2021 - Laura Bouchard  
  
  
Since launching our YouTube channel in October, we have created 25+ instructional videos on how to utilize our API connectors in Power Automate. The goal of the videos is to provide comprehensive, real-time walkthroughs for the wide range of functions available through the connectors. If you are unfamiliar with our API connectors, we have a great lineup available in Power Automate and Logic Apps that reflect similar categories to our APIs, e.g. Document Conversion, Data Validation, Virus Scan, Natural Language Processing, etc. I’ve provided a varied selection of video examples below that you can incorporate into your workflows to improve productivity and efficiency.

### Check them out here:

[How to Convert an Excel File to JSON in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Convert-an-Excel-File-to-JSON-in-Power-Automate-and-Logic-Apps) [How to Get PDF Metadata in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Get-PDF-Metadata-in-Power-Automate-and-Logic-Apps) [How to Virus Scan and Quarantine a File in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Virus-Scan-and-Quarantine-a-File-in-Power-Automate-and-Logic-Apps) [How to Generate a QR Code in Power Automate and Logic Apps](https://cloudmersive.com/videos/How-to-Generate-a-QR-Code-in-Power-Automate-and-Logic-Apps) [How to Convert Word Documents to PDF in Power Automate and Logic Apps](https://cloudmersive.com/videos/Convert-Word-Documents-to-PDF-in-Power-Automate)  
  
  
  
[Customizable Protection: Cloudmersive Reverse Proxy Server Policies](/blog/Customizable-Protection%3a-Cloudmersive-Reverse-Proxy-Server-Policies)  
5/4/2021 - Laura Bouchard  
  
  
In December, we introduced you to our Virus Scanning Reverse Proxy service; as a reminder, this no-code solution automatically protects web applications from threats before they get a chance to reach your server. In today’s blog post, I am going to dive a bit deeper into the solution by providing a brief overview of the security policies that you can set up according to your company’s needs.

### Virus Scan File Uploads (mulitpart/form-data) – Regular or Advanced Scan

Our most popular security policy, application of this setting will automatically scan any file uploads to your site via multipart/form-data and block the request from passing to the target server if the request is contaminated. Target threat types include viruses and malware, with the option to allow/block executables, scripts, macros, invalid files, password-protected files, and specific file types as well.

### Virus Scan JSON Binary Data – Regular or Advanced Scan

Application of this policy will allow you to automatically scan base-64 encoded binary file data in JSON requests. Additional options include setting a URL Match Regular Expression, setting a URL for a virus found notification page or error page, and configuring specific JSON fields to virus scan by specifying a JSON path.

### SQL Injection Protection

The SQLI protection policy will guard your web application from SQL Injection attacks by automatically scanning text inputs.

### XXE Protection

Similar to the SQLI protection policy, the XXE protection policy will automatically scan text inputs for XML External Entity attacks.

### XSS Protection for Request Parameters

This policy enables you to block Cross-Site Scripting (XSS) attack requests.

### IP Blocklist and IP Allowlist

The IP Blocklist policy will allow you to block access from any IP addresses specified on the blocklist, and the IP Allowlist policy will allow you to block access from any IP addresses NOT specified on the allowlist. The latter policy is primarily used for internal services/APIs.

### Rate Limit

Configure a Rate Limit policy to automatically block client IP addresses that exceed the defined rate limit; this is achieved by defining the rate limit value and unit of time (per second/per minute).

### Block Known Bot, Threat, or Tor Clients

The titles of these three policies are basically self-explanatory; you can choose to block known bot clients, threat clients, or Tor clients by configuring any or all of the policies. If you have questions or would like more information, you can contact our knowledgeable team, who are always happy to help.  
  
  
  
[New Validation APIs to Up your Security Game](/blog/New-Validation-APIs-to-Up-your-Security-Game)  
4/27/2021 - Laura Bouchard  
  
  
It seems that the list of online threats is constantly growing, which is why our engineers at Cloudmersive are continually researching and developing new APIs and services to protect you from all angles. This week, I’m excited to share four new data validation APIs that have been designed to scan for threats that fly under the radar of basic anti-virus software. These functions have been uploaded and are available for use in our [API Console](https://api.cloudmersive.com/swagger/index.html?urls.primaryName=Validate%20API).

### Scan Text for Structured Query Language (SQL) Injection Attacks

We have added two new APIs that will scan a single text input or multiple text inputs in batch for SQL injection attacks; these attacks target security vulnerabilities within website or database form fields and have become a serious online threat to many companies. Integration of these APIs will allow you to automatically detect an SQLI attack and define the threat detection level you want to utilize; set it to Normal to target a high-security SQL Injection detection level with a very low false positive rate, or select High to target a very-high security SQL Injection detection level with higher false positives. Default is Normal (recommended).

### Check Text for XML External Entity (XXE) Attacks

Similar to the above APIs, our next validation APIs were also designed to scan a single text input or multiple text inputs in batch. However, these functions target XXE attacks, which are a type of cyber-threat that exploits an opening that occurs when an application parses an XML request to create an output. In addition to scanning the input text for XXE attacks, this API includes optional parameters to block internet-based dependency URLs such as DTDs or create lists of safe/un-safe URLs. Keep an eye out for our next update which will be coming soon! If you have questions about these APIs or any of our other services, you can reach out to our expert team, who will be happy to assist.  
  
  
  
[Cloudmersive Storage Protect: Azure Blob Edition](/blog/Cloudmersive-Storage-Protect%3a-Azure-Blob-Edition)  
4/23/2021 - Laura Bouchard  
  
  
In our previous blog post, I shared how you can use our Cloudmersive Storage Protect service for AWS S3. Now – as promised – I’m going to discuss how to use it to protect files uploaded to an Azure Blob.

### Process Overview

Similar to the AWS S3 process, the Cloudmersive Storage Protection system integrates with your Azure Blob system to apply security policies for all files that are uploaded to a Blob container. Notifications will be sent when new files are created or updated, and the service will automatically respond with the necessary action.

### Flexible Deployment

Again, deployment for your Azure Blob storage will be very similar to the AWS S3 options. You can choose to use Cloudmersive Storage Protection in a Managed Instance or in a Private Cloud self-managed deployment. For both models, you will need to contact your Cloudmersive rep to provision the required instance or licenses.

### Connection and Setup

The setup for your Azure Blob service is very simple and can be completed within 5 minutes. The first step is to navigate to the Cloudmersive Account Portal to add a connection by filling in the required fields and specifying the outcome actions you want to occur when Clean or Infected files are identified. Then you will configure the callback from Azure Storage to Cloudmersive Storage Protect by managing the connection you created in the previous step, adding your API key, and following the instructions provided to setup the Azure Blob Storage notifications to your Cloudmersive server. Want more details? We created a video on our YouTube channel which showcases the process step-by-step. Check it out here: [How to Scan Azure Blob Files for Viruses](https://youtu.be/Bj7_wKLkMI8)  
  
  
  
[Introducing Cloudmersive Storage Protect for AWS S3](/blog/Introducing-Cloudmersive-Storage-Protect-for-AWS-S3)  
4/16/2021 - Laura Bouchard  
  
  
Have you checked out our Cloudmersive Storage Protect service yet? It allows you to protect your cloud storage from viruses, malware, and other threats by automatically scanning files and objects uploaded to AWS S3 or Azure - no coding required! In this post, I’m going to give you a quick rundown of what the process looks like for AWS S3 and how the setup works.

### Seamless Integration

The Cloudmersive Storage Protection system integrates with your AWS S3 system, allowing you to apply security policies for all files that are uploaded to AWS S3. You’ll get notified when new files are created or updated, and the service will automatically respond with the necessary action.

### Flexible Deployment

You can choose to use Cloudmersive Storage Protect in a Managed Instance or in a Private Cloud self-managed deployment. For both options, you will need to contact your Cloudmersive rep to provision the required instance or licenses.

### Get Connected

Setup for this service is super quick and efficient. First, you’ll create a secure cloud connection between Cloudmersive Storage Protect and AWS S3; you can specify the outcome actions for what you want to happen when Clean or Infected files are found, allowing for a fully customized experience. After that, all you’ll need to do is connect AWS S3 back to Cloudmersive Storage Protect by managing the connection you created and setting up notifications to the Cloudmersive server. Keep an eye out for our next blog post, which will discuss how to use Cloudmersive Storage Protect for Azure Blob! If you have any questions, reach out to our excellent sales team, who are always happy to help.  
  
  
  
[We've Added New Programming Languages to our Library](/blog/We%27ve-Added-New-Programming-Languages-to-our-Library)  
4/8/2021 - Laura Bouchard  
  
  
Here at Cloudmersive, we are constantly exploring opportunities to enhance our product line and improve the overall experience for our customers. In the previous blog post, I hinted that we had some exciting new capabilities added, and now I am pleased to announce that five new programming languages are available across our entire library of APIs! The additional languages have already been uploaded to our API Console, so you can check them out at your convenience.

### JavaScript

One of the most popular programming languages out there, JavaScript was a no-brainer addition to the library. Along with HTML and CSS, it’s a core online technology and an essential piece of web applications.

### Swift

Developed by Apple and the open-source community as an updated and more secure replacement for Objective-C, Swift is one of the fastest-growing programming languages.

### C/C++

C++ was created as an extension of the C programming language, and while they are very similar, they do have some significant differences. For the APIs, we have consolidated them under one option, and the developer can specify which language they want to use.

### Go

Go is a programming language that was designed by Google to improve programming productivity and provide an alternative that took some of the positives from other programming languages and left out the negatives.

### cURL

Client URL (cURL) is a little different than the above additions, but valuable nonetheless; it is a computer software project which provides a library and command-line tool that allows you to transfer data via various network protocols.  
  
  
  
[New AllowMacros Feature Added to Advanced Virus Scan API](/blog/New-AllowMacros-Feature-Added-to-Advanced-Virus-Scan-API)  
3/23/2021 - Laura Bouchard  
  
  
Macros have two sides; on one hand they can be incredibly helpful for automating repetitive tasks and saving time, but on the other hand, they have a history of being the vehicles for malicious content distribution online. To guard against the less desirable side of macros, we have added a new AllowMacros feature to our Advanced Virus Scan API.

### Advanced Virus Scan

As a reminder, our Advanced Virus Scan API provides 360-degree content protection with customizable rules to meet your business’s needs. The AllowMacros feature joins several other parameters such as AllowExecutables, AllowScripts, and AllowInvalidFiles to assist in protecting you and your customers. To activate these features, you simply set the parameter to false to block the potential threat.

### Broad Content Security

If set to false, the AllowMacros component will shield you from threats embedded in document files such as Word, Excel, and PowerPoint embedded macros, as well as other embedded content threats.

### Stay Tuned

We have been adding new APIs and programming languages over the past few weeks, so we will be sharing info about those soon! Keep checking back for updates and if you have questions, never hesitate to contact our knowledgeable Sales team.  
  
  
  
[Cloudmersive Meets Mendix: A New and Exciting Partnership](/blog/Cloudmersive-Meets-Mendix%3a-A-New-and-Exciting-Partnership)  
1/14/2021 - Laura Bouchard  
  
  
We are pleased to share that the Cloudmersive API Suite is now available on Mendix, a low-code platform that rapidly creates applications to serve business needs. Implementation and sign-up for a free account can be done in just five minutes; once complete, you will have access to all the Cloudmersive APIs with one plan/pricing model and end-to-end support and services.

### How It Works

The Cloudmersive Mendix Connector takes some of our most popular APIs and uses visual development tools to enable quick results with minimal manual coding.

### Key Features

  * Scan files for viruses, generate PDF/Word files, convert documents, resize/manipulate images
  * Seamless integration in your project with reusable components
  * Example workflows and use cases to enable smart automation of your app
  * Easily extendable new features



### Current Offerings

  * Virus scan
  * Image recognition/processing
  * Generation of Word Docs/PDFs based on templates
  * Content moderation for images and files with not-safe-for-work content



### Future Offerings

  * Document & Data Conversion – automated file conversion for all popular file formats
  * Receipt scanning and business data extraction from photos
  * Malware scan for website URLs
  * Convert/encode/transcode videos
  * Speech-to-text



### Superior Security

When using the Cloudmersive Mendix Connector, you will receive the same top-notch security that you would when using our APIs as a stand-alone. Payload data is not stored or retained after each transaction is completed, which provides you with optimal security, privacy, and performance. For more information, check out our [Cloudmersive Security](https://www.cloudmersive.com/security) page.  
  
  
  
[Our Newest Service: Virus Scanning Reverse Proxy Server ](/blog/Our-Newest-Service%3a-Virus-Scanning-Reverse-Proxy-Server-)  
12/9/2020 - Katherine Coyle  
  
  
We are excited to announce the introduction of our newest service, the Virus Scanning Reverse Proxy Server. This feature can be used to automatically protect any web application or API from viruses and malware, with no code changes.

### Scan and Block Malware

Stop malware uploads before they reach your server, and protect against viruses in base-64 binary data in JSON and XML APIs.

### Wide Protection with No Code Changes

With a single deployment and easy management, you can utilize virus and malware protection across all your applications with no need for code changes.

### Advanced Protection and Mission Critical Performance

With advanced virus scanning capabilities, this feature can scan for over 17 million virus and malware signatures, provide 360-degree content verification, and ensure multi-factor threat analysis. All of this will be partnered with high availability, disaster recovery, and multi-node and multi-datacenter region deployments to ensure mission critical infrastructure and performance. If you’d like to learn more, contact our expert team, who are always happy to help.  
  
  
  
[Take a Look at our New Data Validation APIs](/blog/Take-a-Look-at-our-New-Data-Validation-APIs)  
12/1/2020 - Katherine Coyle  
  
  
We know how important it is to have a polished, professional platform available for use by your organization and users. This means automating and simplifying as much as possible to create a streamlined, easy-to-use site that provides all necessary data without having to perform any external queries. We are making this even more attainable by introducing five new data validation APIs to our library that will retrieve and verify information for location and time.

### Normalize a Street Address

This function will take a structured street address, parse it, and return its parts as separate, normalized objects including the latitude and longitude of the location.

### Get the Current Date and Time

With this, you can return the current date and time for your location. Its response is synchronized with atomic clocks so it will always be consistent and accurate.

### Retrieve a list of all Public Holidays

This will enumerate a list of all the public holidays for a given country in a specified year, with support for over 100 countries.

### Parse a Structured Date and Time Object

With this function, you can parse a structured date and time string into a date/time object. This is intended for standardized date strings that adhere to formatting conventions, rather than natural language input.

### Use NLP to Parse Unstructured Date/Time Objects

Unlike the function above, this API will parse unstructured date/time strings. It is intended for lightweight human-entered input, such as “tomorrow at 3pm” or "Tuesday," and will be verified using Natural Language Processing to retrieve the structured time information for the input.  
  
  
  
[We’ve Added New File Conversion APIs to our Library](/blog/We%e2%80%99ve-Added-New-File-Conversion-APIs-to-our-Library)  
11/16/2020 - Katherine Coyle  
  
  
I am excited to announce that we have recently updated one of our most popular offerings, our Conversion APIs! These updates include new functions that will help you automate and optimize your business such as Data File Conversion, PDF Editing and Conversion, and Document Conversion.

# Convert Data Files

### CSV to XML, XLSX to XML

We have added two new APIs for converting data to XML from CSV and XLSX. XML is a universal file format that is older and more widely used that JSON. This format is computer-readable and thus it is similar to HTML easily processed by online systems. However, it is not as constrained, as it is designed for storing any data type.

# Edit PDF Documents

### PDF to PDF/A

This API will convert PDF files to PDF/A-1b or PDF/A-2b standardized documents. PDF/A is an internationally standardized form of PDF in that it meets universal compliance and compatibility standards that may not be possible with the original PDF file type. Because of this, a PDF/A file will look the same across platforms while a PDF might become corrupted or face formatting errors.

### PDF Reduce File Size

This function will optimize the content of a PDF document to minimize its file size. This is useful when scanning and sharing contracts or other signed documents, as the original PDF file may be too large to email. With this function, you can reduce the size of the file by digitally compressing its contents.

### PDF Linearize

With this API, you can linearize the content of a PDF file to optimize it for streaming download, particularly over web streaming. By performing the function, your PDF file will be simplified internally, with the contents stored in the correct order for viewing. Thus, large files can be viewed while its contents are downloaded to improve your overall efficiency.

# Revert Office Documents to Legacy Format

### DOCX to DOC, XLSX to XLS, PPTX to PPT

For those who work with older versions of the Microsoft Office Suite, or who have partners that maintain usage of these programs, these new APIs will give you the capability to downgrade your Word (DOCX), Excel (XLSX), and PowerPoint (PPTX) documents to their legacy formats: DOC, XLS, and PPT.  
  
  
  
[Introducing Our New “How To” YouTube Channel ](/blog/Introducing-Our-New-%e2%80%9cHow-To%e2%80%9d-YouTube-Channel-)  
10/15/2020 - Katherine Coyle  
  
  
To better assist you in utilizing our top-tier APIs, we have just launched our new “How To” channel over on YouTube! The goal for this channel is to provide comprehensive, real-time walkthroughs for our functions in programs like Power Automate, Azure Logic Apps, and more. Ultimately, adding these to your workflow will not only vastly increase overall working speed but also radically boost your business’ digital capabilities with a few clicks of a button. We already have a few videos posted for some of our Power Automate connectors.

### Check them out here:

[How to Convert an Excel Spreadsheet to PDF in Power Automate and Logic Apps](https://youtu.be/HLpXgeK8H7A) [How to Convert a Single Excel Worksheet to PDF in Power Automate and Logic Apps](https://youtu.be/74fjeeEyJJA) [How to Convert a CSV File to JSON in Power Automate and Logic Apps](https://youtu.be/kLPLdXq5V98) [How to Convert a JPG Image to PNG in Power Automate and Logic Apps](https://youtu.be/R4gCRik8VgQ)  
  
  
  
[Upgrade Security with our New IP Address and Domain Name APIs ](/blog/Upgrade-Security-with-our-New-IP-Address-and-Domain-Name-APIs-)  
10/5/2020 - Katherine Coyle  
  
  
The security of your organization and your clients’ information is always at the forefront of our innovations. Thus, we are pleased to announce our newest IP Address and Domain Name APIs. These updates are all based around increasing your organization’s online security and protecting you from any known or unknown threats. Go check out our [API Console](https://api.cloudmersive.com/swagger/index.html?urls.primaryName=Validate%20API) where these functions are now up and available.

### Scan for Known IP Address Threats

Boost your site’s security by allowing our IP Threat Detection API to scan any incoming IP Addresses for known botnets, bad IPs, compromised servers, and other lists of threats.

### Tor Exit Node Server Detection

This function can detect if an input IP Address is a Tor exit node server. This will let you know if, and when, a user’s IP address is hidden behind a privacy wall.

### Return Domain Name Quality Score

This new Domain Name API can be used to check the quality of a domain name. These scores indicate the authority of a domain name, with high scores equating to more trustworthy sites. Keep checking back here as we continue to add to and improve our library of APIs. If you’d like to learn more, contact our expert team, who are always happy to help.  
  
  
  
[Introducing our Address Validation APIs! ](/blog/Introducing-our-Address-Validation-APIs!-)  
9/25/2020 - Katherine Coyle  
  
  
At Cloudmersive we are working constantly to increase your business capabilities through advanced, class-leading API technology. Because of that, we are always looking for new ways to add to our extensive library of APIs. Thus, we are excited to introduce our new, secure [Address Validation APIs](https://cloudmersive.com/validate-api)! These functions are available in our API Console now, so go take a look and check out all of their new capabilities.

### Parse Addresses

You can now input unstructured text strings and our function will use Natural Language Processing (NLP) to return a formatted address with support for international and other formatting norms.

### Validate Address Information

If you need to extract specific data from an input address, our APIs will parse and give select output on:

  * Street Address
  * City
  * State
  * Postal code
  * Country
  * Regional Information

This capability has support for all major international addresses and will return the latitude and longitude of the location.

### Populate a List of Countries

Our Country List API will now allow you to enumerate a list of ISO 3166-1 countries including country codes, currency, region, and EU Membership.

### Check European Union Membership

For those who do business internationally, we now have a function specifically for validating EU membership for an input address. This will aid in following GDPR and other EU specific regulations and cut down on hassle during international transactions.

### Retrieve Currency, Region, and Time Zone Information

If you are solely looking for regional data from an address, a few of our new APIs will extract currency, continent, and time zone data directly to cut down on time and improve your workflow. Make sure to keep an eye out for our next update so that you can stay at the cutting-edge of the API world. If you’d like to learn more about our constantly updating library of APIs, you can contact our team of experts, who are always happy to help!  
  
  
  
[Our Newest Service - Currency and Exchange Rate APIs](/blog/Our-Newest-Service---Currency-and-Exchange-Rate-APIs)  
9/21/2020 - Katherine Coyle  
  
  
Here at Cloudmersive, we aim to offer you class-leading, enterprise-grade solutions with our secure APIs. With that in mind, we are excited to announce some of our newest API offerings (available now!), [Currency APIs](https://cloudmersive.com/currency-api)! Within this category, we are introducing three different APIs that will enable you to easily convert prices and track and display current exchange rate data.

### Convert Prices Between 30+ Currencies

This API will allow you to convert prices between international currencies automatically at your Point of Sale. Along with the price conversion, this process will return the appropriate currency symbol and ISO currency code.

### Exchange Rate Data

Within this category, we have two APIs that will provide you with constantly updating exchange rate data. The first will populate a list of all available currencies along with their currency symbol, country of origin, and EU membership status, while the second will specify the exchange rates between specific source and destination currencies. With these top-of-the-line API solutions, you can upgrade your online commerce operations and improve your productivity in an instant! For inquests concerning this or one of our many other APIs, you can contact our Cloudmersive team for help.  
  
  
  
[New Cloudmersive Enterprise Management Capabilities](/blog/New-Cloudmersive-Enterprise-Management-Capabilities)  
9/17/2020 - Katherine Coyle  
  
  
In our continued efforts to optimize your Cloudmersive experience through our secure, enterprise-grade API platform, we have rolled out a new feature on your Account Dashboard! When logging into your Cloudmersive Account, check for our latest update, the [Security Center](https://account.cloudmersive.com/security). With this feature, you now have direct access to monitor your API Keys and usage as well as account security measures and more. This page also features two new tools for our partners on Business level and above, Two-Factor Authentication management and an API Analytics viewer.

### Manage Two-Factor Authentication

Our Two-Factor Authentication helps protect your information by adding increased security to your account.

### View API Analytics

The API Analytics viewer allows review of all individual API calls made through your Cloudmersive account to help you manage your end-to-end performance. This tool will also track your granular API-level access logs and provide you with real-time analytics. The next time you access your Cloudmersive account, make sure to check out these new features and see how we are working to keep you protected and moving forward. Our team is available to help, so contact your Cloudmersive representative to learn more about everything Cloudmersive has to offer.  
  
  
  
[Video and Media Services API Released](/blog/Video-and-Media-Services-API-Released)  
8/10/2020 - Juanzi Li  
  
  
We are excited to announce that the [Cloudmersive Video and Media Services API](https://cloudmersive.com/video-and-media-services-api) has just been released to general availability. This powerful new API enables rapid conversion, encoding, moderation, editing and processing of video and audio content at scale.

### Video Conversion and Encoding

Programmatically convert and encode videos on-demand with our powerful video encoding pipeline APIs. Leverage key formats such as MP4, MOV, WEBM, and GIF. Our high-performance APIs rapidly process input files. Get great results by default, or customize and fine-tune encoding parameters.

### Audio Conversion and Encoding

Use a similar approach for the most widely used audio formats in the industry as well. Programmatically convert and encode audio on-demand with our powerful video encoding pipeline APIs. Leverage key formats such as MP3, M4A, WAV and AAC.

### Video Editing

Programmatically edit and split video streams, with support for a wide range of video formats.

### Video Resize

Programmatically resize video streams, with or without aspect ratio preservation, with support for a wide range of video formats.

### Video Thumbnail Extraction

Programmatically extract thumbnails from input videos.

### Automated Video Moderation

Automatically scan input videos for moderation criteria, such as nudity and not-safe-for-work (NSFW) content using our advanced Deep Learning APIs.

### Metadata Extraction

Programmatically extract data from existing videos, including format, size, type.  
  
  
  
[New Document Conversion APIs Released](/blog/New-Document-Conversion-APIs-Released)  
8/3/2020 - Juanzi Li  
  
  
We are excited to announce the availability of a number of key new Document Conversion and Processing APIs as well as some key improvements.

### New DOCX and PPTX Replace All APIs

We released new APIs to replace single or multiple strings in DOCX and PPTX files. This makes it very easy to use a DOCX or PPTX as a template, and fill in dynamic data through multiple string replacements.

### New HTML Edit APIs

We released new HTML APIs to quickly and easily build HTML documents and reports. These documents can then be used together with our HTML to PDF, HTML to image, or HTML to DOCX APIs to build reports and images quickly and easily.

### Improvements to HTML to PDF APIs

We enhanced our HTML to PDF APIs to support scale factor as a parameter, and the option to render background graphics.

### New CSV and XLSX to HTML APIs

We released new APIs that enable CSV to HTML, XLSX to HTML, and CSV to PDF.  
  
  
  
[File Processing Connector for PowerAutomate and Azure Logic Apps Released](/blog/File-Processing-Connector-for-PowerAutomate-and-Azure-Logic-Apps-Released)  
8/1/2020 - Juanzi Li  
  
  
We are excited to announce the availability of the Cloudmersive File Processing Connector for Microsoft Power Automate, Azure Logic Apps, and Microsoft PowerApps worldwide. This connector enables the processing of files at scale for a wide array of business use cases.

### Zip File Processing

Create, compress, and extract zip files. Add encryption or decryption, and password protection.

### Text Processing

Encode and decode text, process and parse text, and generate text output for reporting. Manage whitespace, line endings, find and replace and more.  
  
  
  
[We Are Hiring - Technical Writer](/blog/We-Are-Hiring---Technical-Writer)  
5/12/2020 - Juanzi Li  
  
  
Are you passionate about building great technical documentation and articles for developers and businesses? Then this post is for you. Come join our talented team of technical writers doing great work. We are excited to announce a new technical writer position at Cloudmersive. [Apply today](https://cloudmersive.com/careers/technical-writer) through our website.  
  
  
  
[COVID 19 Response](/blog/COVID-19-Response)  
3/3/2020 - Juanzi Li  
  
  
We are reaching out to update you on Cloudmersive's response to the COVID-19 pandemic. We are committed to keeping communication lines open and strong during this time of global uncertainty. We are committed to keeping the safety and wellbeing of our employees and customers our number one priority. We understand that many of our customers are experiencing great disruptions to their work and personal lives. We are here for you during this time of need.

### Commitment to Continued Service

Cloudmersive's number one value is trust. Our Disaster Recovery and Business Continuity Teams have been carefully monitoring global changes as they occur to keep us informed and prepared. Our incredible teams of dedicated employees are working to ensure guaranteed continued service. We understand that you rely on us to help your business run, and we are committed to keeping you informed. At this time, we do not expect COVID-19 to affect the quality of your service. We will, of course, continue to keep you informed.

### Cloudmersive Employee Safety

Our adaptive team members will continue to support you from the safety of their homes, while continuing to keep your security and service needs a top priority. Cloudmersive is committed to the safety of our employees and our communities. By keeping our employees home and suspending all business travel until further notice, we are doing our part to help flatten the curve.

### Here to Support You

Cloudmersive remains committed to ensuring customer success. We understand that your needs may change as the world adapts to COVID-19. We are dedicated to maintaining customer success through the duration of this crisis. Our communication lines are open, and we welcome you to let us know if your needs change during this trying time. Should you have any questions or concerns please feel free to reach out to our support team. We are always happy to answer any questions you may have. Thank you for your support and understanding. The Cloudmersive Team  
  
  
  
[We Are Hiring - Senior Software Engineer](/blog/We-Are-Hiring---Senior-Software-Engineer)  
1/15/2020 - Juanzi Li  
  
  
Are you passionate about building amazing API products for leading businesses all the around the world? Then this message is for you. We are excited to announce that we are hiring for a Senior Software Engineer position. This position offers key perks such as remote working, and a high level of responsibility within Cloudmersive. [Apply today](https://cloudmersive.com/careers/senior-software-engineer) through our website.  
  
  
  
  
#### Related

[Videos](/videos)  
[Release Notes](/release-notes)  
[Technical Articles](/article)  
[Company](/about)  
[Customers](/customers)  
[Careers](/careers)  
[Documentation](https://api.cloudmersive.com)  
  
  
### 800 free API calls/month, with no expiration

[Get started now! __](https://portal.cloudmersive.com/signup.aspx)or[![Sign in with Google](/images/btn_google_signin_light_pressed_web.png)](https://login.cloudmersive.com/Account/Login?StartGoogle=login)

### API Products

### [![](/images/check.svg) Validate APIs](/validate-api)

### [![](/images/message-circle.svg) Natural Language Processing (NLP) APIs](/nlp-api)

### [![](/images/aperture.svg) Optical Character Recognition (OCR) APIs](/ocr-api)

### [![](/images/zap.svg) Barcode APIs](/barcode-api)

### [![](/images/eye.svg) Image and Face Recognition and Processing APIs](/image-recognition-and-processing-api)

### [![](/images/shield.svg) Virus Scan APIs](/virus-api)

### [![](/images/activity.svg) Security Threat Detection APIs](/security-threat-detection-api)

### [![](/images/file.svg) Document and Data Conversion APIs](/convert-api)

### Questions? We'll be your guide.

[Contact Sales __](https://portal.cloudmersive.com/contact-sales)

### About Us

Cloudmersive® is a leader in Highly Scalable Cloud APIs.

[Sign Up __](https://portal.cloudmersive.com/signup.aspx)

[ 1-415-234-7421](tel:1-415-234-7421)  
9 AM - 5 PM Eastern Time 

4695 Chabot Drive, Suite 200  
Pleasanton, California 94588  
United States

1-415-276-2094

### Sales

  * [Contact Sales](https://portal.cloudmersive.com/contact-sales)
  * [Why Cloudmersive](/why-cloudmersive)
  * [Private Cloud](/private-cloud)
  * [Resellers](https://portal.cloudmersive.com/resellers)
  * [Partnerships](https://portal.cloudmersive.com/partnerships)



### Help & Support

  * [Contact Support](https://portal.cloudmersive.com/contact-support)
  * [Documentation](https://api.cloudmersive.com)
  * [FAQ & KB](/faq)
  * [Examples & Try It](https://api.cloudmersive.com/swagger/index.html?urls.primaryName=Convert%20API)



### Company

  * [Customers](/customers)
  * [About](/about)
  * [Security](/security)
  * [Data Center Regions](/regions)
  * [Careers](https://cloudmersive.com/careers/default)
  * [Blog](/blog)



### Our APIs

  * [Natural Language Processing](/nlp-api)


  * [Data Validation](/validate-api)


  * [OCR](/ocr-api)


  * [Barcode](/barcode-api)


  * [Image Recognition and Processing](/image-recognition-and-processing-api)


  * [Virus Scan](/virus-api)


  * [Document and Data Conversion](/convert-api)



  * [Terms of Service](https://portal.cloudmersive.com/terms-of-service.aspx)
  * [Privacy Policy](/privacy-policy.aspx)
  * [DPA](/data-processing-dpa.aspx)



(C) 2025 Cloudmersive, LLC. All Rights Reserved. Cloudmersive® is a registered trademark of Cloudmersive, LLC.

All product names, logos, and brands are property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, logos, and brands does not imply endorsement.

**Business Email** **Name** **Phone Number** Talk to Sales


scrape_date: 2025-03-14T16:01:05.296091
content_type: markdown
hash: d426343ed2520247f7a6f479e78454ab3b11dc6897f9fcd3c9a258e764a0c5fd
url:
https://stackoverflow.com/questions/28035490/in-python-how-can-i-get-the-next-and-previous-keyvalue-of-a-particular-key-in

title:
In Python, How can I get the next and previous key:value of a particular key in a dictionary? - Stack Overflow

content:
Skip to main content

[ Stack Overflow ](https://stackoverflow.com)

  1. [About](https://stackoverflow.co/)
  2. Products 
  3. [OverflowAI](https://stackoverflow.co/teams/ai/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav-bar&utm_content=overflowai)



  1. [ Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers ](https://stackoverflow.co/teams/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=stack-overflow-for-teams)
  2. [ Advertising & Talent Reach devs & technologists worldwide about your product, service or employer brand ](https://stackoverflow.co/advertising/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=stack-overflow-advertising)
  3. [ OverflowAI GenAI features for Teams ](https://stackoverflow.co/teams/ai/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=overflow-ai)
  4. [ OverflowAPI Train & fine-tune LLMs ](https://stackoverflow.co/api-solutions/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=overflow-api)
  5. [ Labs The future of collective knowledge sharing ](https://stackoverflow.co/labs/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=labs)
  6. [About the company](https://stackoverflow.co/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=about-the-company) [Visit the blog](https://stackoverflow.blog/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=top-nav&utm_content=blog)



Loading…

  1. ###  [current community](https://stackoverflow.com)

     * [ Stack Overflow  ](https://stackoverflow.com)

[help](https://stackoverflow.com/help) [chat](https://chat.stackoverflow.com/?tab=site&host=stackoverflow.com)

     * [ Meta Stack Overflow  ](https://meta.stackoverflow.com)

###  your communities 

[Sign up](https://stackoverflow.com/users/signup?ssrc=site_switcher&returnurl=https%3a%2f%2fstackoverflow.com%2fquestions%2f28035490%2fin-python-how-can-i-get-the-next-and-previous-keyvalue-of-a-particular-key-in) or [log in](https://stackoverflow.com/users/login?ssrc=site_switcher&returnurl=https%3a%2f%2fstackoverflow.com%2fquestions%2f28035490%2fin-python-how-can-i-get-the-next-and-previous-keyvalue-of-a-particular-key-in) to customize your list. 

### [more stack exchange communities](https://stackexchange.com/sites)

[company blog](https://stackoverflow.blog)

  2.   3. [Log in](https://stackoverflow.com/users/login?ssrc=head&returnurl=https%3a%2f%2fstackoverflow.com%2fquestions%2f28035490%2fin-python-how-can-i-get-the-next-and-previous-keyvalue-of-a-particular-key-in)
  4. [Sign up](https://stackoverflow.com/users/signup?ssrc=head&returnurl=https%3a%2f%2fstackoverflow.com%2fquestions%2f28035490%2fin-python-how-can-i-get-the-next-and-previous-keyvalue-of-a-particular-key-in)



  1.     1. [ Home ](/)
    2. [ Questions ](/questions)
    3. [ Tags ](/tags)
    4.     5. [ Users ](/users)
    6. [ Companies ](https://stackoverflow.com/jobs/companies?so_medium=stackoverflow&so_source=SiteNav)
    7. [ Labs ](javascript:void\(0\))
    8. [ Jobs ](/jobs?source=so-left-nav)
    9. [ Discussions ](/beta/discussions)
    10. [ Collectives ](javascript:void\(0\))

    11. Communities for your favorite technologies. [Explore all Collectives](/collectives-all)

  2. Teams

![](https://cdn.sstatic.net/Img/teams/teams-promo.svg?v=e507948b81bf)

Ask questions, find answers and collaborate at work with Stack Overflow for Teams. 

[Try Teams for free](https://stackoverflowteams.com/teams/create/free/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=side-bar&utm_content=explore-teams) [Explore Teams](https://stackoverflow.co/teams/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=side-bar&utm_content=explore-teams)

  3. [ Teams ](javascript:void\(0\))
  4. Ask questions, find answers and collaborate at work with Stack Overflow for Teams. [Explore Teams](https://stackoverflow.co/teams/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=side-bar&utm_content=explore-teams-compact)




##### Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

[ Learn more about Collectives ](/collectives)

**Teams**

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

[ Learn more about Teams ](https://stackoverflow.co/teams/)

Get early access and see previews of new features.

[ Learn more about Labs](https://stackoverflow.co/labs/)

# [In Python, How can I get the next and previous key:value of a particular key in a dictionary?](/questions/28035490/in-python-how-can-i-get-the-next-and-previous-keyvalue-of-a-particular-key-in)

[ Ask Question ](/questions/ask)

Asked 10 years, 1 month ago

Modified [3 years, 1 month ago](?lastactivity "2022-01-29 11:17:58Z")

Viewed 77k times 

33 

[](/posts/28035490/timeline "Show activity on this post.")

Okay, so this is a little hard to explain, but here goes:

I have a dictionary, which I'm adding content to. The content is a hashed username (key) with an IP address (value). I was putting the hashes into an order by running them against base 16, and then using Collection.orderedDict. So, the dictionary looked a little like this:
    
    
    d = {'1234': '8.8.8.8', '2345':'0.0.0.0', '3213':'4.4.4.4', '4523':'1.1.1.1', '7654':'1.3.3.7', '9999':'127.0.0.1'}
    

What I needed was a mechanism that would allow me to pick one of those keys, and get the key/value item one higher and one lower. So, for example, If I were to pick 2345, the code would return the key:value combinations '1234:8.8.8.8' and '3213:4.4.4.4'

So, something like:
    
    
    for i in d:
      while i < len(d)
       if i == '2345':
         print i.nextItem
         print i.previousItem
         break()
    

  * [python](/questions/tagged/python "show questions tagged 'python'")
  * [dictionary](/questions/tagged/dictionary "show questions tagged 'dictionary'")
  * [key](/questions/tagged/key "show questions tagged 'key'")



[Share](/q/28035490 "Short permalink to this question")

[Improve this question](/posts/28035490/edit)

Follow 

[edited Jan 20, 2015 at 3:26](/posts/28035490/revisions "show all edits to this post")

Brewer

asked Jan 19, 2015 at 23:43

[![Brewer's user avatar](https://www.gravatar.com/avatar/065ce4affdbfb588f5a8e25af9117008?s=64&d=identicon&r=PG&f=y&so-version=2)](/users/4366879/brewer)

[Brewer](/users/4366879/brewer)Brewer

51111 gold badge55 silver badges1313 bronze badges

16

  * 10

Dicts are not ordered. There is no next or previous item.

- [Aran-Fey](/users/1222951/aran-fey "43,366 reputation")

Commented Jan 19, 2015 at 23:44

  * 1

I have mine ordered using 'collections.orderedDict'. If There isn't a way of doing this in a dict, can I take the keys out, and put them into a list (or tuple, or whatever), and then search for the next and previous item, and then use that to re-query the dictionary for their values?

- [Brewer](/users/4366879/brewer "511 reputation")

Commented Jan 19, 2015 at 23:49

  * 1

`collections.OrderedDict` objects have a list of 2-tuples internally. If you iterate through them using `.items()` you should get them in the order they are added.

- [metatoaster](/users/2904896/metatoaster "18,978 reputation")

Commented Jan 19, 2015 at 23:51

  * 9

If you're using an `OrderedDict`, you should **definitely mention that** , in the question itself.

- [jonrsharpe](/users/3001761/jonrsharpe "122,169 reputation")

Commented Jan 19, 2015 at 23:51

  * 1

@jamylak OP was already using a `for` to iterate through the keys, might as well iterate through that using `items()`. As you noted, question is misleading and unclear hence I just left a comment to hopefully guide him to figure something out.

- [metatoaster](/users/2904896/metatoaster "18,978 reputation")

Commented Jan 20, 2015 at 2:25




 |  Show **11** more comments

##  11 Answers 11

Sorted by:  [ Reset to default ](/questions/28035490/in-python-how-can-i-get-the-next-and-previous-keyvalue-of-a-particular-key-in?answertab=scoredesc#tab-top)

Highest score (default)  Trending (recent votes count more)  Date modified (newest first)  Date created (oldest first) 

12 

[](/posts/28035607/timeline "Show activity on this post.")

**Edit:** OP now states that they are using OrderedDicts but the use case still requires this sort of approach.

Since dicts are not ordered you cannot directly do this. From your example, you are trying to reference the item like you would use a linked list.

A quick solution would be instead to extract the keys and sort them then iterate over that list:
    
    
    keyList=sorted(d.keys())
    for i,v in enumerate(keyList):
        if v=='eeee':
            print d[keyList[i+1]]
            print d[keyList[i-1]]
    

The `keyList` holds the order of your items and you have to go back to it to find out what the next/previous key is to get the next/previous value. You also have to check for i+1 being greater than the list length and i-1 being less than 0.

You can use an OrderedDict similarly but I believe that you still have to do the above with a separate list as OrderedDict doesn't have next/prev methods.

[Share](/a/28035607 "Short permalink to this answer")

[Improve this answer](/posts/28035607/edit)

Follow 

answered Jan 19, 2015 at 23:56

[![Adam Kerz's user avatar](https://i.sstatic.net/Rehw1.jpg?s=64)](/users/3537579/adam-kerz)

[Adam Kerz](/users/3537579/adam-kerz)Adam Kerz

96111 gold badge66 silver badges1515 bronze badges




Add a comment  | 

8 

[](/posts/28036114/timeline "Show activity on this post.")

As seen in the [`OrderedDict` source code](https://hg.python.org/cpython/file/2.7/Lib/collections.py#l69), if you have a key and you want to find the next and prev in O(1) here's how you do that.
    
    
    >>> from collections import OrderedDict
    >>> d = OrderedDict([('aaaa', 'a',), ('bbbb', 'b'), ('cccc', 'c'), ('dddd', 'd'), ('eeee', 'e'), ('ffff', 'f')])
    >>> i = 'eeee'
    >>> link_prev, link_next, key = d._OrderedDict__map['eeee']
    >>> print 'nextKey: ', link_next[2], 'prevKey: ', link_prev[2]
    nextKey:  ffff prevKey:  dddd
    

This will give you next and prev by insertion order. If you add items in random order then just keep track of your items in sorted order. 

[Share](/a/28036114 "Short permalink to this answer")

[Improve this answer](/posts/28036114/edit)

Follow 

[edited Jan 20, 2015 at 10:42](/posts/28036114/revisions "show all edits to this post")

answered Jan 20, 2015 at 0:56

[![jamylak's user avatar](https://i.sstatic.net/M1ZlR.jpg?s=64)](/users/1219006/jamylak)

[jamylak](/users/1219006/jamylak)jamylak

134k3030 gold badges237237 silver badges239239 bronze badges

4

  * 8

That's not good practice - __map is a private attribute and is not guaranteed to exist or continue to function the same way in different versions or different platforms.

- [Adam Kerz](/users/3537579/adam-kerz "961 reputation")

Commented Jan 20, 2015 at 4:17

  * 1

@AdamKerz It's the only way to access the `OrderedDict` linked list if you need it for whatever purpose, it doesn't matter, you need this if you want to find the `next` and `prev` in O(1). Keep in mind I answered the actual question, and the title was misleading for the actual problem OP is having

- [jamylak](/users/1219006/jamylak "133,754 reputation")

Commented Jan 20, 2015 at 10:43

  * 1

I never said that it doesn't work, or that there's another way to access `OrderedDict`'s linked list, just that it's not good practice. The OP never mentioned O(1) complexity, and yet my solution is also generally O(1) complexity (assuming dict lookup is O(1) or close as it is in normal cases).

- [Adam Kerz](/users/3537579/adam-kerz "961 reputation")

Commented Jan 21, 2015 at 4:15

  * @AdamKerz Yes I agree it is bad practise if the logic of the program can be redesigned this should not be needed. Your solution is never O(1) because you sort the keys.

- [jamylak](/users/1219006/jamylak "133,754 reputation")

Commented Mar 12, 2020 at 3:28




Add a comment  | 

3 

[](/posts/28035905/timeline "Show activity on this post.")

You could also use the `list.index()` method.

This function is more generic (you can check positions +n and -n), it will catch attempts at searching a key that's not in the dict, and it will also return `None` if there's nothing before of after the key:
    
    
    def keyshift(dictionary, key, diff):
        if key in dictionary:
            token = object()
            keys = [token]*(diff*-1) + sorted(dictionary) + [token]*diff
            newkey = keys[keys.index(key)+diff]
            if newkey is token:
                print None
            else:
                print {newkey: dictionary[newkey]}
        else:
            print 'Key not found'
    
    
    keyshift(d, 'bbbb', -1)
    keyshift(d, 'eeee', +1)
    

[Share](/a/28035905 "Short permalink to this answer")

[Improve this answer](/posts/28035905/edit)

Follow 

[edited Jan 20, 2015 at 10:41](/posts/28035905/revisions "show all edits to this post")

answered Jan 20, 2015 at 0:30

[![Roberto's user avatar](https://www.gravatar.com/avatar/e530f9ba4d11f9a1694a900a72ac5dad?s=64&d=identicon&r=PG&f=y&so-version=2)](/users/2962364/roberto)

[Roberto](/users/2962364/roberto)Roberto

2,7862121 silver badges3232 bronze badges

1

  * This one is good as well, i think the usage should be considered, if the access to prev/next element will happen very often or how often the inserts to the dictionary will happen.

- [user1767754](/users/1767754/user1767754 "25,164 reputation")

Commented Jan 20, 2015 at 0:43




Add a comment  | 

3 

[](/posts/58505904/timeline "Show activity on this post.")

Another way that seems simple and straight forward: this function returns the key which is _offset_ positions away from _k_
    
    
    def get_shifted_key(d:dict, k:str, offset:int) -> str:
        l = list(d.keys())
        if k in l:
            i = l.index(k) + offset
            if 0 <= i < len(l):
                return l[i]
        return None    
    

[Share](/a/58505904 "Short permalink to this answer")

[Improve this answer](/posts/58505904/edit)

Follow 

answered Oct 22, 2019 at 13:53

[![Florian Metzger-Noel's user avatar](https://i.sstatic.net/0euFe.jpg?s=64)](/users/3448078/florian-metzger-noel)

[Florian Metzger-Noel](/users/3448078/florian-metzger-noel)Florian Metzger-Noel

9521313 silver badges3030 bronze badges




Add a comment  | 

2 

[](/posts/28035679/timeline "Show activity on this post.")

Try:
    
    
    pos = 0
    d = {'aaaa': 'a', 'bbbb':'b', 'cccc':'c', 'dddd':'d', 'eeee':'e', 'ffff':'f'}
    
    for i in d:
        pos+=1
        if i == 'eeee':
            listForm = list(d.values())
            print(listForm[pos-1])
            print(listForm[pos+1])
    

As in @AdamKerz's answer `enumerate` seems pythonic, but if you are a beginner this code might help you understand it in an easy way.

And I think its faster + smaller compared to sorting followed by building list & then enumerating

[Share](/a/28035679 "Short permalink to this answer")

[Improve this answer](/posts/28035679/edit)

Follow 

[edited Jan 20, 2015 at 0:09](/posts/28035679/revisions "show all edits to this post")

answered Jan 20, 2015 at 0:04

[![RinkyPinku's user avatar](https://www.gravatar.com/avatar/7d0c62fcb0d7fdf01560dbd4f3c043d2?s=64&d=identicon&r=PG&f=y&so-version=2)](/users/4453945/rinkypinku)

[RinkyPinku](/users/4453945/rinkypinku)RinkyPinku

41044 silver badges2020 bronze badges

1

  * Note this only works if you actually use an OrderedDict... :-) And you are also creating a list there: list(d.values()). And possibly multiple lists if the conditional matches more than one key (no, it won't in this example). Note that sorted does build a list (but that is necessary if you are interested in alphabetical order rather than insert order (as was the original request)). And finally, enumerate returns an iterator so that is basically the same in terms of efficiency as what you are doing and less verbose.

- [Adam Kerz](/users/3537579/adam-kerz "961 reputation")

Commented Jan 20, 2015 at 4:28




Add a comment  | 

2 

[](/posts/28036101/timeline "Show activity on this post.")

You could use a generic function, based on iterators, to get a moving window (taken from [this](https://stackoverflow.com/questions/6822725/rolling-or-sliding-window-iterator-in-python) question):
    
    
    import itertools
    
    def window(iterable, n=3):
        it = iter(iterable)
        result = tuple(itertools.islice(it, n))
        if len(result) == n:
            yield result
        for element in it:
            result = result[1:] + (element,)
            yield result
    
    l = range(8)
    for i in window(l, 3):
        print i
    

Using the above function with `OrderedDict.items()` will give you three (key, value) pairs, in order:
    
    
    d = collections.OrderedDict(...)
    
    for p_item, item, n_item in window(d.items()):
        p_key, p_value = p_item
        key, value = item
        # Or, if you don't care about the next value:
        n_key, _ = n_item
    

Of course using this function the first and last values will never be in the middle position (although this should not be difficult to do with some adaptation).

I think the biggest advantage is that it does not require table lookups in the previous and next keys, and also that it is generic and works with any iterable.

[Share](/a/28036101 "Short permalink to this answer")

[Improve this answer](/posts/28036101/edit)

Follow 

[edited May 23, 2017 at 12:19](/posts/28036101/revisions "show all edits to this post")

[![Community's user avatar](https://www.gravatar.com/avatar/a007be5a61f6aa8f3e85ae2fc18dd66e?s=64&d=identicon&r=PG)](/users/-1/community)

[Community](/users/-1/community)Bot

111 silver badge

answered Jan 20, 2015 at 0:54

[![Rafael Lerm's user avatar](https://www.gravatar.com/avatar/0fdfda92bbbb1ea1d55ae504b4246e88?s=64&d=identicon&r=PG)](/users/340862/rafael-lerm)

[Rafael Lerm](/users/340862/rafael-lerm)Rafael Lerm

1,40077 silver badges1010 bronze badges




Add a comment  | 

2 

[](/posts/28035979/timeline "Show activity on this post.")

Maybe it is an overkill, but you can keep Track of the Keys inserted with a Helper Class and according to that list, you can retrieve the Key for Previous or Next. Just don't forget to check for border conditions, if the objects is already first or last element. This way, you will not need to always resort the ordered list or search for the element.
    
    
    from collections import OrderedDict
    
    class Helper(object):
        """Helper Class for Keeping track of Insert Order"""
        def __init__(self, arg):
            super(Helper, self).__init__()
    
        dictContainer = dict()
        ordering = list()
    
        @staticmethod
        def addItem(dictItem):
            for key,value in dictItem.iteritems():
                print key,value
                Helper.ordering.append(key)
                Helper.dictContainer[key] = value
    
        @staticmethod
        def getPrevious(key):
            index = (Helper.ordering.index(key)-1)
            return Helper.dictContainer[Helper.ordering[index]]
    
    
    #Your unordered dictionary
    d = {'aaaa': 'a', 'bbbb':'b', 'cccc':'c', 'dddd':'d', 'eeee':'e', 'ffff':'f'}
    
    #Create Order over keys
    ordered = OrderedDict(sorted(d.items(), key=lambda t: t[0]))
    
    #Push your ordered list to your Helper class
    Helper.addItem(ordered)
    
    
    #Get Previous of    
    print Helper.getPrevious('eeee')
    >>> d
    

[Share](/a/28035979 "Short permalink to this answer")

[Improve this answer](/posts/28035979/edit)

Follow 

[edited Jan 20, 2015 at 8:39](/posts/28035979/revisions "show all edits to this post")

answered Jan 20, 2015 at 0:38

[![user1767754's user avatar](https://www.gravatar.com/avatar/00ba1fd17dc5970a0096cc4d7a932536?s=64&d=identicon&r=PG)](/users/1767754/user1767754)

[user1767754](/users/1767754/user1767754)user1767754

25.2k2020 gold badges161161 silver badges175175 bronze badges




Add a comment  | 

1 

[](/posts/28038627/timeline "Show activity on this post.")

You can store the keys and values in temp variable in prior, and can access previous and next key,value pair using index. 

It is pretty dynamic, will work for any key you query. Please check this code :
    
    
    d = {'1234': '8.8.8.8', '2345':'0.0.0.0', '3213':'4.4.4.4', '4523':'1.1.1.1', '7654':'1.3.3.7', '9999':'127.0.0.1'}
    ch = raw_input('Pleasure Enter your choice : ')
    keys = d.keys()
    values = d.values()
    #print keys, values
    for k,v in d.iteritems():
        if k == ch:
            ind = d.keys().index(k)
            print keys[ind-1], ':',values[ind-1]
            print keys[ind+1], ':',values[ind+1]
    

[Share](/a/28038627 "Short permalink to this answer")

[Improve this answer](/posts/28038627/edit)

Follow 

answered Jan 20, 2015 at 5:53

[![user123's user avatar](https://www.gravatar.com/avatar/45bb1f11796208e1ca968c1c4ae22240?s=64&d=identicon&r=PG)](/users/2518644/user123)

[user123](/users/2518644/user123)user123

5,4071616 gold badges7878 silver badges125125 bronze badges




Add a comment  | 

0 

[](/posts/51802667/timeline "Show activity on this post.")

I think this is a nice Pythonic way of resolving your problem using a [lambda](https://docs.python.org/3/tutorial/controlflow.html?highlight=lambda#lambda-expressions) and [list comprehension](https://docs.python.org/3/library/stdtypes.html?highlight=list%20comprehension#lists), although it may not be optimal in execution time:
    
    
    import collections
    
    x = collections.OrderedDict([('a','v1'),('b','v2'),('c','v3'),('d','v4')])
    
    previousItem = lambda currentKey, thisOrderedDict : [
        list( thisOrderedDict.items() )[ z - 1 ] if (z != 0) else None
        for z in range( len( thisOrderedDict.items() ) )
        if (list( thisOrderedDict.keys() )[ z ] == currentKey) ][ 0 ]
    
    nextItem = lambda currentKey, thisOrderedDict : [
        list( thisOrderedDict.items() )[ z + 1 ] if (z != (len( thisOrderedDict.items() ) - 1)) else None
        for z in range( len( thisOrderedDict.items() ) )
        if (list( thisOrderedDict.keys() )[ z ] == currentKey) ][ 0 ]
    
    assert previousItem('c', x) == ('b', 'v2')
    assert nextItem('c', x) == ('d', 'v4')
    assert previousItem('a', x) is None
    assert nextItem('d',x) is None
    

[Share](/a/51802667 "Short permalink to this answer")

[Improve this answer](/posts/51802667/edit)

Follow 

answered Aug 11, 2018 at 18:37

[![blueskyjunkie's user avatar](https://www.gravatar.com/avatar/c81b067e7b2ff4c5b230086d306396f7?s=64&d=identicon&r=PG)](/users/1633716/blueskyjunkie)

[blueskyjunkie](/users/1633716/blueskyjunkie)blueskyjunkie

7122 silver badges77 bronze badges




Add a comment  | 

0 

[](/posts/68144986/timeline "Show activity on this post.")

i know how to get next key:value of a particular key in a dictionary:
    
    
    flag = 0
    for k, v in dic.items():
        if flag == 0:
            code...
            flag += 1
            continue
        code...{next key and value in for}
    

[Share](/a/68144986 "Short permalink to this answer")

[Improve this answer](/posts/68144986/edit)

Follow 

answered Jun 26, 2021 at 17:52

[![mahdi gadget's user avatar](https://lh3.googleusercontent.com/-S2FWnaRQuz4/AAAAAAAAAAI/AAAAAAAAAAA/ACHi3ref6_1X4faMe9dE4jF-g3GdtoXh6w/photo.jpg?sz=64)](/users/14179419/mahdi-gadget)

[mahdi gadget](/users/14179419/mahdi-gadget)mahdi gadget

11311 gold badge11 silver badge66 bronze badges

1

  * Welcome to stackoverflow! Unfortunately, it's not so clear to me what your answer adds to the existing accepted answer. Moreover, your code does not run. It would be helpful to have full working code to understand what you're suggesting here, and perhaps also explain why your solution might be better than some of the other solutions posted. This might be helpful too: [stackoverflow.com/help/how-to-answer](https://stackoverflow.com/help/how-to-answer) Good luck!

- [Matthias C. M. Troffaes](/users/2863746/matthias-c-m-troffaes "605 reputation")

Commented Jun 27, 2021 at 8:02




Add a comment  | 

0 

[](/posts/70904774/timeline "Show activity on this post.")

if correct :
    
    
    d = { "a": 1, "b":2, "c":3 }
    l = list( d.keys() )  # make a list of the keys
    k = "b"               # the actual key
    i = l.index( k )      # get index of the actual key
    

for the next :
    
    
    i = i+1 if i+1 < len( l ) else 0        # select next index or restart 0
    n = l [ i ]
    d [ n ]
    

for the previous :
    
    
    i = i-1 if i-1 >= 0 else len( l ) -1    # select previous index or go end
    p = l [ i ]
    d [ p ]
    

[Share](/a/70904774 "Short permalink to this answer")

[Improve this answer](/posts/70904774/edit)

Follow 

answered Jan 29, 2022 at 11:17

[![s4mdf0o1's user avatar](https://i.sstatic.net/dxZpu.png?s=64)](/users/1800755/s4mdf0o1)

[s4mdf0o1](/users/1800755/s4mdf0o1)s4mdf0o1

46866 silver badges88 bronze badges




Add a comment  | 

##  Your Answer 

Thanks for contributing an answer to Stack Overflow!

  * Please be sure to _answer the question_. Provide details and share your research!



But _avoid_ …

  * Asking for help, clarification, or responding to other answers.
  * Making statements based on opinion; back them up with references or personal experience.



To learn more, see our [tips on writing great answers](/help/how-to-answer).

Draft saved

Draft discarded

### Sign up or [log in](/users/login?ssrc=question_page&returnurl=https%3a%2f%2fstackoverflow.com%2fquestions%2f28035490%2fin-python-how-can-i-get-the-next-and-previous-keyvalue-of-a-particular-key-in%23new-answer)

Sign up using Google 

Sign up using Email and Password 

Submit

### Post as a guest

Name

Email

Required, but never shown

### Post as a guest

Name

Email

Required, but never shown

Post Your Answer  Discard 

By clicking “Post Your Answer”, you agree to our [terms of service](https://stackoverflow.com/legal/terms-of-service/public) and acknowledge you have read our [privacy policy](https://stackoverflow.com/legal/privacy-policy).

Start asking to get answers

Find the answer to your question by asking.

[Ask question](/questions/ask)

Explore related questions

  * [python](/questions/tagged/python "show questions tagged 'python'")
  * [dictionary](/questions/tagged/dictionary "show questions tagged 'dictionary'")
  * [key](/questions/tagged/key "show questions tagged 'key'")



See similar questions with these tags.

  * The Overflow Blog 
  * [Can climate tech startups address the current crisis?](https://stackoverflow.blog/2025/03/14/can-climate-tech-startups-address-the-current-crisis/)

  * [What we learned at TDX 2025](https://stackoverflow.blog/2025/03/14/what-we-learned-at-tdx-2025/)

  * Featured on Meta 
  * [Community Asks Sprint Announcement - March 2025](https://meta.stackexchange.com/questions/407321/community-asks-sprint-announcement-march-2025)

  * [Meta Stack Exchange site maintenance scheduled starting Monday, March 17,...](https://meta.stackexchange.com/questions/407425/meta-stack-exchange-site-maintenance-scheduled-starting-monday-march-17-2025-f "Meta Stack Exchange site maintenance scheduled starting Monday, March 17, 2025 from 13:30 UTC to 14:30 UTC \(9:30am to 10:30am ET\)")

  * [Policy: Generative AI (e.g., ChatGPT) is banned](https://meta.stackoverflow.com/questions/421831/policy-generative-ai-e-g-chatgpt-is-banned)

  * [Stacks Editor development and testing](https://meta.stackoverflow.com/questions/433034/stacks-editor-development-and-testing)

  * [Is it better to redirect users who attempt to perform actions they can't yet...](https://meta.stackoverflow.com/questions/432989/is-it-better-to-redirect-users-who-attempt-to-perform-actions-they-cant-yet-tak "Is it better to redirect users who attempt to perform actions they can't yet take?")




#### Linked

[ 225 ](/q/6822725 "Question score \(upvotes - downvotes\)") [Rolling or sliding window iterator?](/questions/6822725/rolling-or-sliding-window-iterator?noredirect=1)

[ -2 ](/q/58264132 "Question score \(upvotes - downvotes\)") [Get next and previous item in dictionary using Python](/questions/58264132/get-next-and-previous-item-in-dictionary-using-python?noredirect=1)

#### Related

[ 0 ](/q/18653191 "Question score \(upvotes - downvotes\)") [Is there a way to get next value for the same key in the next dictionary in the list of dictionaries?](/questions/18653191/is-there-a-way-to-get-next-value-for-the-same-key-in-the-next-dictionary-in-the)

[ 2 ](/q/19992078 "Question score \(upvotes - downvotes\)") [How to access in OrderedDict in Python previous key and value?](/questions/19992078/how-to-access-in-ordereddict-in-python-previous-key-and-value)

[ 1 ](/q/46920002 "Question score \(upvotes - downvotes\)") [Python Dictionary: Accessing the previous value using current/present key](/questions/46920002/python-dictionary-accessing-the-previous-value-using-current-present-key)

[ 1 ](/q/51870964 "Question score \(upvotes - downvotes\)") [Fetching values with consecutive keys from dictionary](/questions/51870964/fetching-values-with-consecutive-keys-from-dictionary)

[ 0 ](/q/52209156 "Question score \(upvotes - downvotes\)") [Python : Check specific key in nested dictionary and return all previous keys if its matched](/questions/52209156/python-check-specific-key-in-nested-dictionary-and-return-all-previous-keys-if)

[ 0 ](/q/57211502 "Question score \(upvotes - downvotes\)") [Given a known value in a key-value pair, extract the value of the next key-value pair in a dictionary](/questions/57211502/given-a-known-value-in-a-key-value-pair-extract-the-value-of-the-next-key-value)

[ 1 ](/q/57386642 "Question score \(upvotes - downvotes\)") [How to return to previous value in dictionary given a specific value?](/questions/57386642/how-to-return-to-previous-value-in-dictionary-given-a-specific-value)

[ 3 ](/q/61058709 "Question score \(upvotes - downvotes\)") [Return next key of a given dictionary key, python 3.6+](/questions/61058709/return-next-key-of-a-given-dictionary-key-python-3-6)

[ 0 ](/q/68839838 "Question score \(upvotes - downvotes\)") [How to print the key which comes before the matching key in python dictionaries](/questions/68839838/how-to-print-the-key-which-comes-before-the-matching-key-in-python-dictionaries)

[ 1 ](/q/71807278 "Question score \(upvotes - downvotes\)") [How to find previous element in ordered dict which has a looked up value](/questions/71807278/how-to-find-previous-element-in-ordered-dict-which-has-a-looked-up-value)

####  [ Hot Network Questions ](https://stackexchange.com/questions?tab=hot)

  * [ Why can the time between power on and the beginning of POST vary? ](https://superuser.com/questions/1885628/why-can-the-time-between-power-on-and-the-beginning-of-post-vary)
  * [ Noobie trying to get a turbo trainer ](https://bicycles.stackexchange.com/questions/96363/noobie-trying-to-get-a-turbo-trainer)
  * [ What is the highest temperature the butter can be used for baking at? ](https://cooking.stackexchange.com/questions/130343/what-is-the-highest-temperature-the-butter-can-be-used-for-baking-at)
  * [ Edit Mode Not showing Vertices ](https://blender.stackexchange.com/questions/332307/edit-mode-not-showing-vertices)
  * [ Does the Moon really need its own atomic clocks and timescales, separate from those used on and around Earth? ](https://space.stackexchange.com/questions/68059/does-the-moon-really-need-its-own-atomic-clocks-and-timescales-separate-from-th)
  * [ How to control expansion in \str_if_eq:nnTF ](https://tex.stackexchange.com/questions/738899/how-to-control-expansion-in-str-if-eqnntf)
  * [ Web application contains a link to a non-existing domain, is this a vulnerability? ](https://security.stackexchange.com/questions/280787/web-application-contains-a-link-to-a-non-existing-domain-is-this-a-vulnerabilit)
  * [ These entities are not supported: [GenAiPromptTemplate] ](https://salesforce.stackexchange.com/questions/430810/these-entities-are-not-supported-genaiprompttemplate)
  * [ Must getters return values as is? ](https://softwareengineering.stackexchange.com/questions/456824/must-getters-return-values-as-is)
  * [ "on time" in Chess Jargon ](https://english.stackexchange.com/questions/629859/on-time-in-chess-jargon)
  * [ PTIJ: Talmudic mathematics ](https://judaism.stackexchange.com/questions/148242/ptij-talmudic-mathematics)
  * [ Story Identification: A novel about a failed space colony hundreds of years later ](https://scifi.stackexchange.com/questions/295442/story-identification-a-novel-about-a-failed-space-colony-hundreds-of-years-late)
  * [ Align-like environment in text ](https://tex.stackexchange.com/questions/738944/align-like-environment-in-text)
  * [ Is crypto sniping illegal? ](https://law.stackexchange.com/questions/107790/is-crypto-sniping-illegal)
  * [ Did Trump campaign against gay people? ](https://politics.stackexchange.com/questions/90991/did-trump-campaign-against-gay-people)
  * [ What was the real motivation behind Walter White's decision to keep cooking meth even after securing enough money? ](https://movies.stackexchange.com/questions/124803/what-was-the-real-motivation-behind-walter-white-s-decision-to-keep-cooking-meth)
  * [ Is 223.1.2/24 (sic) correct way to label a subnet? ](https://networkengineering.stackexchange.com/questions/86989/is-223-1-2-24-sic-correct-way-to-label-a-subnet)
  * [ Trying to model a swish of paste ](https://blender.stackexchange.com/questions/332291/trying-to-model-a-swish-of-paste)
  * [ Can someone command a lycanthrope to use its shapechanger action? ](https://rpg.stackexchange.com/questions/214984/can-someone-command-a-lycanthrope-to-use-its-shapechanger-action)
  * [ Why is Erdős' conjecture on arithmetic progressions not discussed much, and is there an active pathway to its resolution? ](https://mathoverflow.net/questions/489375/why-is-erd%c5%91s-conjecture-on-arithmetic-progressions-not-discussed-much-and-is-t)
  * [ Classically radiating black holes vs Quantum mechanically radiating black holes ](https://physics.stackexchange.com/questions/845192/classically-radiating-black-holes-vs-quantum-mechanically-radiating-black-holes)
  * [ Is Oz a real place? ](https://movies.stackexchange.com/questions/124822/is-oz-a-real-place)
  * [ What techniques do writers use to create their characters' personalities? ](https://writing.stackexchange.com/questions/71012/what-techniques-do-writers-use-to-create-their-characters-personalities)
  * [ Remove space before nth root ](https://tex.stackexchange.com/questions/738918/remove-space-before-nth-root)

more hot questions 

[ Question feed ](/feeds/question/28035490 "Feed of this question and its answers")

#  Subscribe to RSS 

Question feed 

To subscribe to this RSS feed, copy and paste this URL into your RSS reader.

![](/posts/28035490/ivc/27b4?prg=005f7f2b-0f5a-463d-8146-e6ffb02939e6)

lang-py

[](https://stackoverflow.com)

##### [Stack Overflow](https://stackoverflow.com)

  * [Questions](/questions)
  * [Help](/help)
  * [Chat](https://chat.stackoverflow.com/?tab=site&host=stackoverflow.com)



##### [Products](https://stackoverflow.co/)

  * [Teams](https://stackoverflow.co/teams/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=footer&utm_content=teams)
  * [Advertising](https://stackoverflow.co/advertising/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=footer&utm_content=advertising)
  * [Talent](https://stackoverflow.co/advertising/employer-branding/?utm_medium=referral&utm_source=stackoverflow-community&utm_campaign=footer&utm_content=talent)



##### [Company](https://stackoverflow.co/)

  * [About](https://stackoverflow.co/)
  * [Press](https://stackoverflow.co/company/press/)
  * [Work Here](https://stackoverflow.co/company/work-here/)
  * [Legal](https://stackoverflow.com/legal)
  * [Privacy Policy](https://stackoverflow.com/legal/privacy-policy)
  * [Terms of Service](https://stackoverflow.com/legal/terms-of-service/public)
  * [Contact Us](/contact)
  * Your Privacy Choices 
  * [Cookie Policy](https://stackoverflow.com/legal/cookie-policy)



##### [Stack Exchange Network](https://stackexchange.com)

  * [ Technology ](https://stackexchange.com/sites#technology)
  * [ Culture & recreation ](https://stackexchange.com/sites#culturerecreation)
  * [ Life & arts ](https://stackexchange.com/sites#lifearts)
  * [ Science ](https://stackexchange.com/sites#science)
  * [ Professional ](https://stackexchange.com/sites#professional)
  * [ Business ](https://stackexchange.com/sites#business)
  * [ API ](https://api.stackexchange.com/)
  * [ Data ](https://data.stackexchange.com/)



  * [Blog](https://stackoverflow.blog?blb=1)
  * [Facebook](https://www.facebook.com/officialstackoverflow/)
  * [Twitter](https://twitter.com/stackoverflow)
  * [LinkedIn](https://linkedin.com/company/stack-overflow)
  * [Instagram](https://www.instagram.com/thestackoverflow)



Site design / logo © 2025 Stack Exchange Inc;  user contributions licensed under  [CC BY-SA](https://stackoverflow.com/help/licensing) .  rev 2025.3.14.23880


url: https://arxiv.org/html/2310.14735v5
title: Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review
scrape_date: 2025-03-17T12:00:37.144773
content_type: markdown
hash: 61325241769ad0e740ce4bc6112e0cc72ab1ec8403eef28c58e035c3165c4301
content:
  1. [1 Introduction](https://arxiv.org/html/2310.14735v5#S1 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
  2. [2 Basics of prompt engineering](https://arxiv.org/html/2310.14735v5#S2 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    1. [2.1 Model introduction: GPT-4](https://arxiv.org/html/2310.14735v5#S2.SS1 "In 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    2. [2.2 Giving instructions](https://arxiv.org/html/2310.14735v5#S2.SS2 "In 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    3. [2.3 Be clear and precise](https://arxiv.org/html/2310.14735v5#S2.SS3 "In 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    4. [2.4 Role-prompting](https://arxiv.org/html/2310.14735v5#S2.SS4 "In 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    5. [2.5 Use of triple quotes to separate](https://arxiv.org/html/2310.14735v5#S2.SS5 "In 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    6. [2.6 Try several times](https://arxiv.org/html/2310.14735v5#S2.SS6 "In 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    7. [2.7 One-shot or few-shot prompting](https://arxiv.org/html/2310.14735v5#S2.SS7 "In 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    8. [2.8 LLMs settings: temperature and top-p𝑝pitalic_p](https://arxiv.org/html/2310.14735v5#S2.SS8 "In 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
  3. [3 Advanced methodologies](https://arxiv.org/html/2310.14735v5#S3 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    1. [3.1 Chain-of-thought](https://arxiv.org/html/2310.14735v5#S3.SS1 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      1. [3.1.1 Zero-shot chain-of-thought](https://arxiv.org/html/2310.14735v5#S3.SS1.SSS1 "In 3.1 Chain-of-thought ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      2. [3.1.2 Golden chain-of-thought](https://arxiv.org/html/2310.14735v5#S3.SS1.SSS2 "In 3.1 Chain-of-thought ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    2. [3.2 Self-consistency](https://arxiv.org/html/2310.14735v5#S3.SS2 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    3. [3.3 Generated knowledge](https://arxiv.org/html/2310.14735v5#S3.SS3 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    4. [3.4 Least-to-most prompting](https://arxiv.org/html/2310.14735v5#S3.SS4 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    5. [3.5 Tree of thoughts](https://arxiv.org/html/2310.14735v5#S3.SS5 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    6. [3.6 Graph of thoughts](https://arxiv.org/html/2310.14735v5#S3.SS6 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    7. [3.7 Decomposed prompting](https://arxiv.org/html/2310.14735v5#S3.SS7 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    8. [3.8 Active prompt](https://arxiv.org/html/2310.14735v5#S3.SS8 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    9. [3.9 Prompt pattern catalog](https://arxiv.org/html/2310.14735v5#S3.SS9 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    10. [3.10 Prompt optimization](https://arxiv.org/html/2310.14735v5#S3.SS10 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      1. [3.10.1 Prompt optimization with textual gradients](https://arxiv.org/html/2310.14735v5#S3.SS10.SSS1 "In 3.10 Prompt optimization ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      2. [3.10.2 Black-box prompt optimization](https://arxiv.org/html/2310.14735v5#S3.SS10.SSS2 "In 3.10 Prompt optimization ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      3. [3.10.3 Model-adaptive prompt optimization](https://arxiv.org/html/2310.14735v5#S3.SS10.SSS3 "In 3.10 Prompt optimization ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      4. [3.10.4 PromptAgent](https://arxiv.org/html/2310.14735v5#S3.SS10.SSS4 "In 3.10 Prompt optimization ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      5. [3.10.5 Reinforcement learning](https://arxiv.org/html/2310.14735v5#S3.SS10.SSS5 "In 3.10 Prompt optimization ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      6. [3.10.6 GPTs (plugins)](https://arxiv.org/html/2310.14735v5#S3.SS10.SSS6 "In 3.10 Prompt optimization ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    11. [3.11 Retrieval augmentation](https://arxiv.org/html/2310.14735v5#S3.SS11 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    12. [3.12 Reasoning and active interaction](https://arxiv.org/html/2310.14735v5#S3.SS12 "In 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      1. [3.12.1 Automatic reasoning and tool usage](https://arxiv.org/html/2310.14735v5#S3.SS12.SSS1 "In 3.12 Reasoning and active interaction ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      2. [3.12.2 ReAct framework](https://arxiv.org/html/2310.14735v5#S3.SS12.SSS2 "In 3.12 Reasoning and active interaction ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
  4. [4 Methodologies for multimodal large models](https://arxiv.org/html/2310.14735v5#S4 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    1. [4.1 Zero-shot and few-shot prompting](https://arxiv.org/html/2310.14735v5#S4.SS1 "In 4 Methodologies for multimodal large models ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    2. [4.2 Continuous prompt vectors](https://arxiv.org/html/2310.14735v5#S4.SS2 "In 4 Methodologies for multimodal large models ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    3. [4.3 Context optimization](https://arxiv.org/html/2310.14735v5#S4.SS3 "In 4 Methodologies for multimodal large models ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    4. [4.4 Conditional prompt learning](https://arxiv.org/html/2310.14735v5#S4.SS4 "In 4 Methodologies for multimodal large models ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    5. [4.5 Multimodal prompt learning](https://arxiv.org/html/2310.14735v5#S4.SS5 "In 4 Methodologies for multimodal large models ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
  5. [5 Assessing the efficacy of prompt methods](https://arxiv.org/html/2310.14735v5#S5 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    1. [5.1 Subjective and objective evaluations](https://arxiv.org/html/2310.14735v5#S5.SS1 "In 5 Assessing the efficacy of prompt methods ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      1. [5.1.1 Subjective evaluations](https://arxiv.org/html/2310.14735v5#S5.SS1.SSS1 "In 5.1 Subjective and objective evaluations ‣ 5 Assessing the efficacy of prompt methods ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      2. [5.1.2 Objective evaluations](https://arxiv.org/html/2310.14735v5#S5.SS1.SSS2 "In 5.1 Subjective and objective evaluations ‣ 5 Assessing the efficacy of prompt methods ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    2. [5.2 Comparing different prompt methods](https://arxiv.org/html/2310.14735v5#S5.SS2 "In 5 Assessing the efficacy of prompt methods ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
  6. [6 Applications improved by prompt engineering](https://arxiv.org/html/2310.14735v5#S6 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    1. [6.1 Assessment in teaching and learning](https://arxiv.org/html/2310.14735v5#S6.SS1 "In 6 Applications improved by prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    2. [6.2 Content creation and editing](https://arxiv.org/html/2310.14735v5#S6.SS2 "In 6 Applications improved by prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    3. [6.3 Computer programming](https://arxiv.org/html/2310.14735v5#S6.SS3 "In 6 Applications improved by prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    4. [6.4 Reasoning tasks](https://arxiv.org/html/2310.14735v5#S6.SS4 "In 6 Applications improved by prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    5. [6.5 Dataset generation](https://arxiv.org/html/2310.14735v5#S6.SS5 "In 6 Applications improved by prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
  7. [7 LLMs security](https://arxiv.org/html/2310.14735v5#S7 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    1. [7.1 Adversarial attacks](https://arxiv.org/html/2310.14735v5#S7.SS1 "In 7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      1. [7.1.1 Data poisoning](https://arxiv.org/html/2310.14735v5#S7.SS1.SSS1 "In 7.1 Adversarial attacks ‣ 7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      2. [7.1.2 Backdoor attacks](https://arxiv.org/html/2310.14735v5#S7.SS1.SSS2 "In 7.1 Adversarial attacks ‣ 7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
      3. [7.1.3 Prompt injection and prompt leaking](https://arxiv.org/html/2310.14735v5#S7.SS1.SSS3 "In 7.1 Adversarial attacks ‣ 7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    2. [7.2 Prompt hacking](https://arxiv.org/html/2310.14735v5#S7.SS2 "In 7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    3. [7.3 Model stealing](https://arxiv.org/html/2310.14735v5#S7.SS3 "In 7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    4. [7.4 Enhancing LLMs security](https://arxiv.org/html/2310.14735v5#S7.SS4 "In 7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
  8. [8 Prospective methodologies](https://arxiv.org/html/2310.14735v5#S8 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    1. [8.1 Better understanding of structures](https://arxiv.org/html/2310.14735v5#S8.SS1 "In 8 Prospective methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
    2. [8.2 Agent for AIGC tools](https://arxiv.org/html/2310.14735v5#S8.SS2 "In 8 Prospective methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
  9. [9 Conclusion](https://arxiv.org/html/2310.14735v5#S9 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")
  10. [10 Acknowledgement](https://arxiv.org/html/2310.14735v5#S10 "In Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")



[1]\fnmNicolas \surLangrené [2,1]\fnmShengxin \surZhu

1]Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science, \orgnameBNU-HKBU United International College, \orgaddress\cityZhuhai \postcode519087, \stateGuangdong, \countryChina 2]\orgnameResearch Center for Mathematics, Beijing Normal University, \orgaddress\streetNo.18, Jingfeng Road, \cityZhuhai \postcode519087, \stateGuangdong, \countryChina

# Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review

\fnmBanghao \surChen  [chenbanghao@u.nus.edu](mailto:chenbanghao@u.nus.edu) \fnmZhaofeng \surZhang  [zhangzf@umich.edu](mailto:zhangzf@umich.edu) [nicolaslangrene@uic.edu.cn](mailto:nicolaslangrene@uic.edu.cn) [shengxin.zhu@bnu.edu.cn](mailto:shengxin.zhu@bnu.edu.cn) [  [ 

( First version: October 23, 2023   
This revised version: September 5, 2024   
)

###### Abstract

This comprehensive review delves into the pivotal role of prompt engineering in unleashing the capabilities of Large Language Models (LLMs). The development of Artificial Intelligence (AI), from its inception in the 1950s to the emergence of advanced neural networks and deep learning architectures, has made a breakthrough in LLMs, with models such as GPT-4o and Claude-3, and in Vision-Language Models (VLMs), with models such as CLIP and ALIGN. Prompt engineering is the process of structuring inputs, which has emerged as a crucial technique to maximize the utility and accuracy of these models. This paper explores both foundational and advanced methodologies of prompt engineering, including techniques such as self-consistency, chain-of-thought, and generated knowledge, which significantly enhance model performance. Additionally, it examines the prompt method of VLMs through innovative approaches such as Context Optimization (CoOp), Conditional Context Optimization (CoCoOp), and Multimodal Prompt Learning (MaPLe). Critical to this discussion is the aspect of AI security, particularly adversarial attacks that exploit vulnerabilities in prompt engineering. Strategies to mitigate these risks and enhance model robustness are thoroughly reviewed. The evaluation of prompt methods is also addressed, through both subjective and objective metrics, ensuring a robust analysis of their efficacy. This review also reflects the essential role of prompt engineering in advancing AI capabilities, providing a structured framework for future research and application.

###### keywords: 

prompt engineering, Large Language Models, AI-generated content, adversarial attacks, evaluation, AI agent, GPT-4, Vision-Language Models 

##  1 Introduction

In recent years, a significant milestone in artificial intelligence research has been the progression of natural language processing capabilities, primarily attributed to Large Language Models (LLMs). Many popular models, rooted in the transformer architecture [[1](https://arxiv.org/html/2310.14735v5#bib.bib1)], undergo training on extensive datasets derived from web-based text. Central to their design is a self-supervised learning objective, which focuses on predicting subsequent words in incomplete sentences. Those models are called Artificial Intelligence-Generated Content (AIGC), and their ability to generate coherent and contextually relevant responses is a result of this training process, where they learn to associate words and phrases with their typical contexts.

LLMs operate by encoding the input text into a high-dimensional vector space, where semantic relationships between words and phrases are preserved. The model then decodes this representation to generate a response, guided by the learned statistical patterns [[2](https://arxiv.org/html/2310.14735v5#bib.bib2)]. The quality of the response can be influenced by various factors, including the prompt provided to the model, the model’s hyperparameters, and the diversity of the training data.

These models, including LLMs such as the GPT series [[3](https://arxiv.org/html/2310.14735v5#bib.bib3), [4](https://arxiv.org/html/2310.14735v5#bib.bib4)] by OpenAI, along with many others (e.g. Gemini [[5](https://arxiv.org/html/2310.14735v5#bib.bib5), [6](https://arxiv.org/html/2310.14735v5#bib.bib6)] and Gemini (BARD) [[7](https://arxiv.org/html/2310.14735v5#bib.bib7)] by Google, Claude series by Anthropic [[8](https://arxiv.org/html/2310.14735v5#bib.bib8), [9](https://arxiv.org/html/2310.14735v5#bib.bib9)], and Llama series open-source model from Meta [[10](https://arxiv.org/html/2310.14735v5#bib.bib10), [11](https://arxiv.org/html/2310.14735v5#bib.bib11)]), have revolutionized tasks ranging from information extraction to the creation of engaging content [[12](https://arxiv.org/html/2310.14735v5#bib.bib12)]. In parallel, the development of multimodal large models (MMLMs) has introduced the ability to process and generate not just text, but also images, audio, and other forms of data, showcasing their flexibility and effectiveness. These models integrate multiple data modalities into a single framework, demonstrating strong capabilities in tasks such as image description and visual question answering (VQA). Early MMLMs include the DALL-E series [[13](https://arxiv.org/html/2310.14735v5#bib.bib13), [14](https://arxiv.org/html/2310.14735v5#bib.bib14), [15](https://arxiv.org/html/2310.14735v5#bib.bib15)], which can generate images from textual descriptions, and CLIP, which can understand and relate text and image data in a unified manner [[16](https://arxiv.org/html/2310.14735v5#bib.bib16), [17](https://arxiv.org/html/2310.14735v5#bib.bib17)]. More powerful models such as GPT-4o by OpenAI [[18](https://arxiv.org/html/2310.14735v5#bib.bib18)] and Claude 3.5 Sonnet by Anthropic [[8](https://arxiv.org/html/2310.14735v5#bib.bib8), [9](https://arxiv.org/html/2310.14735v5#bib.bib9)] excel in multimodal tasks involving text generation and understanding, integrating natural language processing with various forms of data to perform diverse and complex tasks. While numerous advanced models are currently capable of processing audio, the majority of accessible Application Programming Interfaces (APIs) remain focused on text and vision modalities. With the gradual introduction of audio APIs, a broad expansion of research in this modality can be expected [[19](https://arxiv.org/html/2310.14735v5#bib.bib19)]. The evolution of LLMs reflects significant strides in AI research, characterized by increasing model complexity, enhanced training methodologies, and broader application potentials. These advancements underline the critical role of prompt engineering in maximizing the utility and accuracy of these models, ensuring that they can effectively cater to diverse and dynamic user needs. While this survey is mainly focused on prompt engineering for LLMs, the inclusion of vision-language models (VLMs) offers a broader perspective, revealing the potential and challenges of prompt engineering in handling multimodal data. By integrating research from both types of models, we can gain a deeper understanding of the applications of prompt engineering and provide valuable insights for future research and practice.

In real applications, the prompt is the input of the model, and prompt engineering can result in significant output differences [[20](https://arxiv.org/html/2310.14735v5#bib.bib20)]. Modifying both the structure (e.g., altering length, arrangement of instances) and the content (e.g., phrasing, choice of illustrations, directives) of the prompt can exert a notable influence on the model’s behavior [[21](https://arxiv.org/html/2310.14735v5#bib.bib21), [22](https://arxiv.org/html/2310.14735v5#bib.bib22)].

Prompt engineering refers to the systematic design and optimization of input prompts to guide the responses of LLMs, ensuring accuracy, relevance, and coherence in the generated output. This process is crucial in harnessing the full potential of these models, making them more accessible and applicable across diverse domains. Over time, prompt engineering has evolved from an empirical practice into a well-structured research domain. As illustrated in Figure [1](https://arxiv.org/html/2310.14735v5#S1.F1 "Figure 1 ‣ 1 Introduction ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"), the historical progression of prompt engineering showcases significant milestones from the early days of structured inputs in the 1950s to advanced methodologies such as chain-of-thought prompting [[23](https://arxiv.org/html/2310.14735v5#bib.bib23)] and self-consistency prompting [[24](https://arxiv.org/html/2310.14735v5#bib.bib24)] developed in recent years. This review will primarily focus on techniques emerging from the period of rapid development after 2017.

![Refer to caption](x1.png) Figure 1: History of the development in prompt engineering.

Contemporary prompt engineering encompasses a spectrum of techniques, ranging from foundational approaches such as role-prompting [[25](https://arxiv.org/html/2310.14735v5#bib.bib25)] to more sophisticated methods such as chain-of-thought prompting [[23](https://arxiv.org/html/2310.14735v5#bib.bib23)]. The domain remains dynamic, with emergent research continually unveiling novel techniques and applications in prompt engineering. The importance of prompt engineering is accentuated by its ability to guide model responses, thereby amplifying the versatility and relevance of LLMs in various sectors. Importantly, a well-constructed prompt can counteract challenges such as machine hallucinations [[26](https://arxiv.org/html/2310.14735v5#bib.bib26), [27](https://arxiv.org/html/2310.14735v5#bib.bib27)]. The influence of prompt engineering extends to numerous disciplines. For instance, it has facilitated the creation of robust feature extractors using LLMs, thereby improving their efficacy in tasks such as defect detection and classification [[28](https://arxiv.org/html/2310.14735v5#bib.bib28)].

This paper aims to provide a comprehensive review of the prompt engineering techniques proposed so far within the realm of LLMs. The structure of the paper is organized as follows: Section [2](https://arxiv.org/html/2310.14735v5#S2 "2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") explores the foundational methods of prompt engineering, emphasizing the importance of clear and precise instructions, role-prompting, and iterative attempts to optimize outputs. In Section [3](https://arxiv.org/html/2310.14735v5#S3 "3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"), advanced methodologies such as chain-of-thought, self-consistency, and generated knowledge are introduced to guide models in generating high-quality content. Section [4](https://arxiv.org/html/2310.14735v5#S4 "4 Methodologies for multimodal large models ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") discusses methodologies specific to VLMs, including Context Optimization (CoOp), Conditional Context Optimization (CoCoOp), and Multimodal Prompt Learning (MaPLe), which enhance the performance of VLMs [[29](https://arxiv.org/html/2310.14735v5#bib.bib29)]. Section [5](https://arxiv.org/html/2310.14735v5#S5 "5 Assessing the efficacy of prompt methods ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") assesses the efficacy of various prompt methods through both subjective and objective evaluations, ensuring a robust analysis of their effectiveness. Section [6](https://arxiv.org/html/2310.14735v5#S6 "6 Applications improved by prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") briefly explores the applications of prompt engineering across diverse fields such as education, content creation, computer programming, and reasoning tasks, highlighting its broad impact. Section [7](https://arxiv.org/html/2310.14735v5#S7 "7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") addresses the security implications of prompt engineering, identifying common vulnerabilities in LLMs and reviewing strategies to enhance security such as adversarial training. Finally, Section [8](https://arxiv.org/html/2310.14735v5#S8 "8 Prospective methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") explores prospective methodologies, emphasizing the importance of understanding AI model structures and the potential of AI agents in advancing AI-generated content tools. This structured framework provides an entire overview of the pivotal role of prompt engineering in advancing AI capabilities and guiding future research and applications.

##  2 Basics of prompt engineering

By incorporating just a few key elements, one can craft a basic prompt that enables LLMs to produce high-quality answers. In this section, some essential components of a well-made prompt will be discussed and examples of these methods will be shown.

###  2.1 Model introduction: GPT-4

All of the examples in the following sections are generated by GPT-4, developed by OpenAI [[4](https://arxiv.org/html/2310.14735v5#bib.bib4)]. Vast amounts of text data have been used to train GPT-4, whose number of parameters has been estimated to be several orders of magnitude larger than the 175 billion parameters that had been used for the earlier GPT-3 [[3](https://arxiv.org/html/2310.14735v5#bib.bib3)]. The architectural foundation of the model rests on transformer [[1](https://arxiv.org/html/2310.14735v5#bib.bib1)], which essentially are attention mechanisms that assign varying weights to input data based on the context. Similar to GPT-3, GPT-4 was also fine-tuned to follow a broad class of written instructions by reinforcement learning from human feedback (RLHF) [[30](https://arxiv.org/html/2310.14735v5#bib.bib30), [31](https://arxiv.org/html/2310.14735v5#bib.bib31)], which is a technique that uses human preferences as a reward signal to fine-tune models.

When GPT-4 receives an input prompt, the input text will be firstly converted into tokens that the model can interpret and process. These tokens are then managed by transformer layers [[32](https://arxiv.org/html/2310.14735v5#bib.bib32)], which capture their relationships and context. Within these layers, attention mechanisms distribute different weights to tokens based on their relevance and context. After attention processing, the model forms its internal renditions of the input data, known as intermediate representations. These representations are then decoded back into human-readable text [[33](https://arxiv.org/html/2310.14735v5#bib.bib33)].

A significant aspect of this process is the randomness function [[32](https://arxiv.org/html/2310.14735v5#bib.bib32)]. This function is influenced by two primary parameters, temperature and top-k𝑘kitalic_k sampling. The first one, temperature [[34](https://arxiv.org/html/2310.14735v5#bib.bib34)] balances the randomness and determinism in the output. A higher temperature value results in more random outputs, while a lower value makes the output more deterministic. The second one, top-k𝑘kitalic_k sampling [[35](https://arxiv.org/html/2310.14735v5#bib.bib35)], limits the model’s choices to the top k𝑘kitalic_k most probable tokens during each step of output generation. The final stage of this process is the output generation, where the model crafts the final text.

###  2.2 Giving instructions

The method of giving instructions, also known as re-reading [[36](https://arxiv.org/html/2310.14735v5#bib.bib36)], refers to the heuristics of human reading strategy. It has been observed that the output generated by GPT-4, introduced in Section [2.1](https://arxiv.org/html/2310.14735v5#S2.SS1 "2.1 Model introduction: GPT-4 ‣ 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"), tends to be excessively general when provided with fundamental instructions devoid of any supplementary description [[37](https://arxiv.org/html/2310.14735v5#bib.bib37), [38](https://arxiv.org/html/2310.14735v5#bib.bib38)]. An example prompt is shown in Figure [2](https://arxiv.org/html/2310.14735v5#S2.F2 "Figure 2 ‣ 2.2 Giving instructions ‣ 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"). When the model is prompted with a basic instruction, it faces a plethora of options, which makes the result quite broad. Hence, a comprehensive description is imperative to elicit more precise and relevant outputs [[39](https://arxiv.org/html/2310.14735v5#bib.bib39)].

I want to understand some cutting-edge aspects of technology.

Here are some of the key areas:   
1\. Artificial intelligence and machine learning: These technologies are becoming increasingly sophisticated, with advancements in areas such as natural language processing (like the technology you’re interacting with right now), computer vision, and reinforcement learning. Techniques such as transfer learning and federated learning are being used to make machine learning models more efficient and less data-hungry.   
2\. Quantum computing: Quantum computers use the principles of quantum mechanics to process information. They hold the potential to solve certain types of problems much more efficiently than classical computers. Companies like Google, IBM, and many startups are investing heavily in this area.   
3\. 5G and beyond: 5G wireless technology began widespread deployment in 2020 and 2021, bringing faster speeds, lower latency, and the ability to connect more devices simultaneously. Research is also being done on 6G, which is expected to integrate even more deeply with AI and potentially bring about more transformative changes.   
4\. Biotechnology: …   


Model inputModel output

Figure 2: Giving instructions without extra description.

###  2.3 Be clear and precise

The second basic prompt method is “to be clear and precise”. This involves formulating prompts that are unambiguous and specific, which can guide the model toward generating the desired output.

Most LLM architectures are derived from an extensive array of textual data. It can be conceptualized as a combination of insights from a myriad of authors. When presented with a broad or undetailed prompt, its output predominantly exhibits a generic nature, which, while being applicable across a range of contexts, may not be optimal for any specific application. In contrast, a detailed and precise prompt enables the model to generate content that is more aligned with the unique requirements of the given scenario, as it reduces the model’s uncertainty and guides it toward the correct response.

For instance, as shown in Figure [3](https://arxiv.org/html/2310.14735v5#S2.F3 "Figure 3 ‣ 2.3 Be clear and precise ‣ 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"), instead of asking a vague requirement such as “I want to understand the cutting edge of technology.”, a more precise prompt would be “I want to understand the cutting edge of technology, specifically related to artificial intelligence and machine learning…”.

I want to understand the cutting edge of technology, specifically related to artificial intelligence and machine learning. Recently, artificial intelligence has become extremely popular, especially the Large Language Models which have amazed me. Please analyze the possible reasons behind the rapid advancement of this technology from at least three perspectives.

Absolutely, the rise of AI and machine learning, especially with regard to Large Language Models like GPT-4, can be seen as a convergence of several key factors. I’ll analyze this from three perspectives: technological advancements, availability of data, and socio-economic demand. 1\. Technological advancements: The growth of AI and machine learning models has been facilitated by several technological innovations. For starters, we’ve seen incredible improvements in computational power, particularly in Graphics Processing Units (GPUs) and now even more specialized AI chips. These advances have made it … 2\. Availability of data: … 3\. Socio-economic demand: …

Model inputModel output

Figure 3: A clearer and more precise prompt.

###  2.4 Role-prompting

Role-prompting is another fundamental method in prompt engineering. It involves giving the model a specific role to play, such as a helpful assistant or a knowledgeable expert [[40](https://arxiv.org/html/2310.14735v5#bib.bib40)]. This method can be particularly effective in guiding the model’s responses and ensuring that they align with the desired output. For instance, if the model is prompted to act as a historian, it is more likely to provide a detailed and contextually accurate response when asked about a historical event [[41](https://arxiv.org/html/2310.14735v5#bib.bib41)]. Another brief example is shown in Figure [4](https://arxiv.org/html/2310.14735v5#S2.F4 "Figure 4 ‣ 2.4 Role-prompting ‣ 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review").

You are an expert in artificial intelligence specializing in Large Language Models…

Here are five basic methods to quickly get started with Large Language Models: 1\. Experiment with web-based interfaces: … 

A role promptModel output

Figure 4: Role prompting example.

###  2.5 Use of triple quotes to separate

In prompt engineering, the use of triple quotes is a technique used to separate different parts of a prompt or to encapsulate multi-line strings. This technique is particularly useful when dealing with complex prompts that include multiple components or when the prompt itself contains quotes, which makes the model understand one’s instructions better [[42](https://arxiv.org/html/2310.14735v5#bib.bib42)].

###  2.6 Try several times

Due to the non-deterministic nature of LLMs, it is often beneficial to try several times when generating responses. This technique, often referred to as “resampling”, involves running the model multiple times with the same prompt and selecting the best output. This approach can help overcome the inherent variability in the model’s responses and increase the chances of obtaining a high-quality output [[34](https://arxiv.org/html/2310.14735v5#bib.bib34)].

###  2.7 One-shot or few-shot prompting

One-shot and few-shot prompting are two important techniques in prompt engineering. One-shot prompting refers to the method where the model is given a single example to learn from, while few-shot [[43](https://arxiv.org/html/2310.14735v5#bib.bib43)] prompting provides the model with multiple examples [[44](https://arxiv.org/html/2310.14735v5#bib.bib44)]. The choice between one-shot and few-shot prompting often depends on the complexity of the task and the capability of the model. For instance, for simple tasks or highly capable models, one-shot prompting might be sufficient. An example is shown in Figure [5](https://arxiv.org/html/2310.14735v5#S2.F5 "Figure 5 ‣ 2.7 One-shot or few-shot prompting ‣ 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"). However, for more complex tasks or less capable models, few-shot prompting can provide additional context and guidance, thereby improving the model’s performance.

Standard Prompt

Q: Yes or no: would a pear sink in water?

A: Yes, a pear would typically float in water, not sink. The reason for this is that pears, like many fruits, are composed primarily of water and have a density that is less than that of water. Objects that are less dense than water will float when placed in water. ×\boldsymbol{\times}bold_×

One-shot Prompt

Q: Yes or no: is it common to see frost during some college commencements?   
A: The answer is yes. Q: Yes or no: would a pear sink in water?

A: The answer is No. \pdfrenderStrokeColor=green,TextRenderingMode=2✓

Figure 5: Comparison of standard prompt and one-shot prompt.

Model inputModel outputModel inputModel output

However, as stated in [[45](https://arxiv.org/html/2310.14735v5#bib.bib45)], “examples don’t always help”, meaning that zero-shot prompting may have better output in some scenarios. Zero-shot prompting [[46](https://arxiv.org/html/2310.14735v5#bib.bib46), [47](https://arxiv.org/html/2310.14735v5#bib.bib47)], in the context of prompt-based learning, involves using a pre-trained LLM to perform tasks without any specific training for those tasks. The model relies on its general knowledge, acquired during pre-training, to generate predictions based on cleverly crafted prompts. This allows the LLMs to handle new tasks with no additional task-specific data, making it adaptable to scenarios with minimal labeled data. [[45](https://arxiv.org/html/2310.14735v5#bib.bib45)] investigated the intricacies of how large generative language models, such as GPT-3, respond to prompts. One of the significant findings from this paper is that zero-shot prompts can, in certain scenarios, outperform few-shot prompts. This suggests that the role of few-shot examples might not be as much about teaching the model a new task (meta-learning) but rather guiding it to recall a task it has already learned. This insight is crucial as it challenges the conventional wisdom that more examples always lead to better performance [[3](https://arxiv.org/html/2310.14735v5#bib.bib3)]. In the context of one-shot or few-shot prompting, it is essential to understand that while examples can guide the model, they do not always enhance its performance. Sometimes, a well-crafted zero-shot prompt can be more effective than providing multiple examples [[48](https://arxiv.org/html/2310.14735v5#bib.bib48)].

###  2.8 LLMs settings: temperature and top-p𝑝pitalic_p

The settings of LLMs, such as the temperature and top-p𝑝pitalic_p, play a crucial role in the generation of responses. The temperature parameter controls the randomness of the generated output: a lower temperature leads to more deterministic outputs [[49](https://arxiv.org/html/2310.14735v5#bib.bib49), [50](https://arxiv.org/html/2310.14735v5#bib.bib50)]. The top-p𝑝pitalic_p parameter, on the other hand, controls the nucleus sampling [[34](https://arxiv.org/html/2310.14735v5#bib.bib34)], which is a method to add randomness to the model’s output [[51](https://arxiv.org/html/2310.14735v5#bib.bib51)]. Adjusting these parameters can significantly affect the quality and diversity of the model’s responses, making them essential tools in prompt engineering. However, it has been noted that certain models, exemplified by ChatGPT, do not permit the configuration of these hyperparameters, barring instances where the Application Programming Interface (API) is employed. [[52](https://arxiv.org/html/2310.14735v5#bib.bib52)] ranks several AI text generators and text-to-image systems in terms of various openness metrics, including the accessibility of their API and model parameters.

##  3 Advanced methodologies

The foundational methods from the previous section can help us produce satisfactory outputs. However, experiments indicate that when using LLMs for complex tasks such as analysis or reasoning, the accuracy of the model’s outputs still has room for improvement. In this section, advanced techniques of prompt engineering will be introduced to guide the model in generating more specific, accurate, and high-quality content.

###  3.1 Chain-of-thought

The concept of “Chain-of-Thought” (CoT) prompting [[23](https://arxiv.org/html/2310.14735v5#bib.bib23)] in LLMs is a relatively new development, which has been shown to significantly improve the accuracy of LLMs on various logical reasoning tasks [[53](https://arxiv.org/html/2310.14735v5#bib.bib53), [54](https://arxiv.org/html/2310.14735v5#bib.bib54), [55](https://arxiv.org/html/2310.14735v5#bib.bib55)]. CoT prompting involves providing intermediate reasoning steps to guide the model’s responses, which can be facilitated through simple prompts such as “Let’s think step by step” or through a series of manual demonstrations, each composed of a question and a reasoning chain that leads to an answer [[56](https://arxiv.org/html/2310.14735v5#bib.bib56), [57](https://arxiv.org/html/2310.14735v5#bib.bib57)]. It also provides a clear structure for the model’s reasoning process, making it easier for users to understand how the model arrived at its conclusions.

[[58](https://arxiv.org/html/2310.14735v5#bib.bib58)] illustrates the application of CoT prompting to medical reasoning, showing that it can effectively elicit valid intermediate reasoning steps from LLMs. [[59](https://arxiv.org/html/2310.14735v5#bib.bib59)] introduces the concept of Self-Education via Chain-of-Thought Reasoning (SECToR), and argues that, in the spirit of reinforcement learning, LLMs can successfully teach themselves new skills by chain-of-thought reasoning. In another study, [[60](https://arxiv.org/html/2310.14735v5#bib.bib60)] used CoT prompting to train verifiers to solve math word problems, demonstrating the technique’s potential in educational applications. [[61](https://arxiv.org/html/2310.14735v5#bib.bib61)] proposed a multimodal version of CoT, called Multimodal-CoT, to handle more complex, multimodal tasks beyond simple text-based tasks, such as visual tasks, further expanding the potential applications of CoT. Furthermore, many works are building upon the CoT framework, for instance, [[62](https://arxiv.org/html/2310.14735v5#bib.bib62)] introduces Automate-CoT, an automated approach for augmenting and selecting rationale chains to enhance the reasoning capabilities of LLMs, thereby reducing the reliance on manually crafted CoT prompts.

####  3.1.1 Zero-shot chain-of-thought

The concept of “Zero-Shot chain-of-thought” (Zero-shot-CoT) prompting is an advanced iteration of the CoT prompting mechanism, where the “zero-shot” aspect implies that the model is capable of performing some reasoning without having seen any examples of the task during training.

[[47](https://arxiv.org/html/2310.14735v5#bib.bib47)] discovered that the augmentation of queries with the phrase “Let’s think step by step” facilitated the generation of a sequential reasoning chain by LLMs. This reasoning chain subsequently proved instrumental in deriving more precise answers. This technique is based on the idea that the model, much like a human, can benefit from having more detailed and logical steps to process the prompt and generate a response.

For instance, the standard prompt is illustrated in Figure [6](https://arxiv.org/html/2310.14735v5#S3.F6 "Figure 6 ‣ 3.1.1 Zero-shot chain-of-thought ‣ 3.1 Chain-of-thought ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"), while the appended phrase, “Let’s think step by step”, is depicted in Figure [7](https://arxiv.org/html/2310.14735v5#S3.F7 "Figure 7 ‣ 3.1.1 Zero-shot chain-of-thought ‣ 3.1 Chain-of-thought ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"). Observations indicate that the incorporation of “Let’s think step by step” enhances the logical coherence and comprehensiveness of the model’s response.

Imagine an infinitely wide entrance, which is more likely to pass through it, a military tank or a car?

Given an “infinitely wide entrance”, both a military tank and a car can pass through it without any constraints due to width. Therefore, neither is more likely than the other based on width alone. Both can pass through with equal likelihood. However, if other factors such as ground conditions, accessibility, purpose, or authorization come into play, the likelihood might change. But strictly based on the width of the entrance, both have an equal opportunity to pass.

Model inputModel output

Figure 6: Standard prompt.

Imagine an infinitely wide entrance, which is more likely to pass through it, a military tank or a car? Let’s think step by step.

Given an infinitely wide entrance, both a military tank and a car could pass through without any issues related to the width. However, there are still several factors that could influence which is more likely to pass through: 1\. Ground surface: tanks are designed to handle rough terrains that cars may struggle with. If the ground leading to and from the entrance is too rugged or steep for a car, the tank would be more likely to pass through. 2\. Weight restrictions: if the entrance has a load limit due to the materials used in its construction, the car, being lighter, could be more likely to pass through. The average car weighs around 1.5 tons, while a military tank can weigh anywhere from 60 to 70 tons. 3\. Height clearance: if the entrance is infinitely wide but not infinitely tall, the height of the vehicle matters. The average height of a car is around 1.5 meters, while tanks can be around 2.4 meters. If the entrance is shorter than around 2.4 meters, a car would be more likely to pass through. 4\. Regulations and permissions: … In conclusion, without specific information about the conditions at the entrance, we would assume that a car is more likely to pass through, simply because cars are more common, more likely to meet potential height and weight restrictions and face fewer regulatory barriers.

Model inputModel output

Figure 7: Adding “Let’s think step by step”.

####  3.1.2 Golden chain-of-thought

[[63](https://arxiv.org/html/2310.14735v5#bib.bib63)] introduced the “golden chain-of-thought”, providing an innovative approach to generating responses to instruction-based queries. This methodology leverages a set of “ground-truth chain-of-thought” solutions incorporated within the prompt, considerably simplifying the task for the model as it circumvents the necessity for independent CoT generation. Concurrently, a novel benchmark comprising detective puzzles has been designed, to assess the abductive reasoning capacities of LLMs, which is also considered an evaluation of the golden CoT. Finally, according to the experiment by [[63](https://arxiv.org/html/2310.14735v5#bib.bib63)], in the context of the golden CoT, GPT-4 exhibits commendable performance, boasting an 83% solve rate of puzzles in contrast to the 38% solve rate of the standard CoT.

However, the characteristics of the Golden CoT requiring the “ground-truth chain-of-thought solutions” as an integral part of the prompt also signifies that the Golden CoT’s contribution to solving such problems is limited, despite its high solve rate of 83%.

###  3.2 Self-consistency

In the assessment of InstructGPT [[64](https://arxiv.org/html/2310.14735v5#bib.bib64)] and GPT-3 [[3](https://arxiv.org/html/2310.14735v5#bib.bib3)] on a new synthetic QA dataset called Proof and Ontology-Generated Question-Answering (PrOntoQA) [[65](https://arxiv.org/html/2310.14735v5#bib.bib65), [66](https://arxiv.org/html/2310.14735v5#bib.bib66)], it was observed that although the most extensive model exhibited capability in reasoning tasks, it encountered challenges in proof planning and the selection of the appropriate proof step amidst multiple options, which caused accuracy uncertainties [[65](https://arxiv.org/html/2310.14735v5#bib.bib65)]. Self-consistency is one of the methods for LLMs to solve this situation, which is an advanced prompting technique that aims to ensure the model’s responses are consistent with each other [[24](https://arxiv.org/html/2310.14735v5#bib.bib24), [23](https://arxiv.org/html/2310.14735v5#bib.bib23)]. This method greatly increases the odds of obtaining highly accurate results. The principle of self-consistency in language models posits that for a complex reasoning problem, there can be multiple reasoning paths leading to the correct answer. In this approach, a language model generates a diverse set of reasoning paths for the same problem. The most accurate and consistent answer is then determined by evaluating and marginalizing across these varied paths, ensuring that the final answer reflects the convergence of multiple lines of thought.

The self-consistency method contains three steps. Firstly, prompt a language model using CoT prompting, then replace the “greedy decode” (1-Best) [[32](https://arxiv.org/html/2310.14735v5#bib.bib32), [67](https://arxiv.org/html/2310.14735v5#bib.bib67)] in CoT prompting by sampling from the language model’s decoder to generate a diverse set of reasoning paths, and finally, marginalize out the reasoning paths and aggregate by choosing the most consistent answer in the final answer set.

It is noteworthy that self-consistency can be harmoniously integrated with most sampling algorithms, including but not limited to, temperature sampling [[49](https://arxiv.org/html/2310.14735v5#bib.bib49), [50](https://arxiv.org/html/2310.14735v5#bib.bib50)], top-k𝑘kitalic_k sampling [[68](https://arxiv.org/html/2310.14735v5#bib.bib68), [69](https://arxiv.org/html/2310.14735v5#bib.bib69), [32](https://arxiv.org/html/2310.14735v5#bib.bib32)], and nucleus sampling [[34](https://arxiv.org/html/2310.14735v5#bib.bib34)]. Nevertheless, such an operation may necessitate the invocation of the model’s API to fine-tune these hyperparameters. In light of this, an alternative approach could be to allow the model to generate results employing diverse reasoning paths, and then generate a diverse set of candidate reasoning paths. The response demonstrating the highest degree of consistency across the various reasoning trajectories is then more inclined to represent the accurate solution [[70](https://arxiv.org/html/2310.14735v5#bib.bib70)].

[[71](https://arxiv.org/html/2310.14735v5#bib.bib71), [2](https://arxiv.org/html/2310.14735v5#bib.bib2)] have shown that self-consistency enhances outcomes in arithmetic, commonsense, and symbolic reasoning tasks. Furthermore, in practice, self-consistency can be combined with other techniques to further enhance the model’s performance. [[72](https://arxiv.org/html/2310.14735v5#bib.bib72)] found that combining self-consistency with a discriminator-guided multi-step reasoning approach significantly improved the model’s reasoning capabilities.

###  3.3 Generated knowledge

The “generated knowledge” [[73](https://arxiv.org/html/2310.14735v5#bib.bib73)] approach in prompt engineering is a technique that leverages the ability of LLMs to generate potentially useful information about a given question or prompt before generating a final response. This method is particularly effective in tasks that require commonsense reasoning, as it allows the model to generate and utilize additional context that may not be explicitly present in the initial prompt.

As exemplified in Figure [6](https://arxiv.org/html/2310.14735v5#S3.F6 "Figure 6 ‣ 3.1.1 Zero-shot chain-of-thought ‣ 3.1 Chain-of-thought ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"), when posing the query to the model, “Imagine an infinitely wide entrance, which is more likely to pass through it, a military tank or a car?”, standard prompts predominantly yield responses that neglect to factor in the “entrance height”. Conversely, as delineated in Figure [8](https://arxiv.org/html/2310.14735v5#S3.F8 "Figure 8 ‣ 3.3 Generated knowledge ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") and Figure [9](https://arxiv.org/html/2310.14735v5#S3.F9 "Figure 9 ‣ 3.3 Generated knowledge ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"), prompting the model to first generate pertinent information and subsequently utilizing generated information in the query leads to outputs with augmented logical coherence and comprehensiveness. Notably, this approach stimulates the model to account for salient factors such as “entrance height”.

Generate two key analyses related to detailed size data on military tanks and cars, and then generate three key influencing factors regarding whether an object can pass through an infinitely wide entrance.

Detailed size analysis: military tanks vs. cars 1\. Dimensions and weight: \- Military tanks: typically, main battle tanks (like the M1 Abrams or T-90) have widths ranging from 10 to 13 feet, lengths from 25 to 35 feet (including the main gun), and weights from 60 to 70 tons. \- Cars: most consumer cars have a width of about 6 to 8 feet, a length of 12 to 20 feet, and weigh between 1 to 3 tons. Some large SUVs and trucks can exceed these figures but are still substantially smaller and lighter than tanks. 2\. Maneuverability & design: \- Military tanks: tanks are designed to traverse rough terrains and have tracks that distribute weight and prevent the vehicle from getting easily stuck in muddy or uneven ground. However, their sheer weight and size limit their maneuverability in urban or congested settings. \- Cars: designed for paved roads, cars are generally more agile and can navigate tight spaces more efficiently than tanks. Their design emphasizes aerodynamics and comfort, unlike tanks that prioritize armor and firepower.   
Three key influencing factors on whether an object can pass through an infinitely wide entrance: 1\. Height of the object: the height is critical. An infinitely wide entrance does not give information about its height. If the object is taller than the entrance, it won’t pass through. 2\. Structural integrity: depending on the material and design of an object, squeezing or forcing it through an entrance could compromise its structural integrity. For instance, certain tanks have reactive armor or sensitive equipment that may be damaged if scraped or bumped forcefully. 3\. Ground clearance and maneuverability: …

Model inputModel output

Figure 8: Generating knowledge (Step1).

“““ Detailed size analysis: military tanks vs. cars 1\. Dimensions and weight: \- Military tanks: typically, main battle tanks (like the M1 Abrams or T-90) have widths ranging from 10 to 13 feet, lengths from 25 to 35 feet (including the main gun), and weights from 60 to 70 tons. \- Cars: most consumer cars have a width of about 6 to 8 feet, a length of 12 to 20 feet, and weigh between 1 to 3 tons. Some large SUVs and trucks can exceed these figures but are still substantially smaller and lighter than tanks. 2\. Maneuverability & design… ”””   
Use the above information to answer the following question: Imagine an infinitely wide entrance, which is more likely to pass through it, a military tank or a car? 

Given the information provided about the dimensions, design, and the factors influencing whether an object can pass through an infinitely wide entrance, we can infer the following: 1\. Height of the object: the height is not specified for either tanks or cars. However, given that tanks are built for rough terrains and often have a turret and other equipment on top, they generally have a taller profile compared to most cars. If the entrance had a restrictive height, cars would generally have an advantage here. 2\. Structural integrity: … 3\. Ground clearance and maneuverability:   
… Conclusion: If the entrance height is not restrictive and the surface is rough, a military tank would be more likely to pass through it without issues. On the other hand, if the entrance is low-height or has tight squeezes that could compromise the structural integrity of larger, robust vehicles, a car might be at an advantage. However, given no height constraints and only the factor of width (since it is “infinitely wide”), both a military tank and a car would easily pass through it. In such a scenario, the distinguishing factor would be the terrain of the entrance. If it is rough or uneven, the tank has the advantage. If it is smooth, both would pass with relative ease.

Model inputModel output

Figure 9: Combining the generated knowledge with the question (Step2).

###  3.4 Least-to-most prompting

The concept of “least-to-most prompting” [[74](https://arxiv.org/html/2310.14735v5#bib.bib74)] is an advanced method that involves decomposing a complex problem into a series of simpler subproblems, which are then addressed sequentially. The foundational premise of this approach is to systematically break down intricate tasks into manageable components. Each subproblem is solved in turn, with the solution to each one serving as a building block for the next. This step-by-step process not only simplifies the problem-solving task for the language model but also enables it to handle more complex challenges than those typically managed by traditional prompting techniques. Figure [10](https://arxiv.org/html/2310.14735v5#S3.F10 "Figure 10 ‣ 3.4 Least-to-most prompting ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") is an illustration of least-to-most prompting applied to a mathematical word. This figure illustrates the least-to-most prompting approach by systematically breaking down a complex problem into a series of simpler subproblems. The process begins with the decomposition of the main problem—calculating the distance a train travels in 2.5 hours—into two sequential subproblems. First, the model is prompted to determine the train’s speed, and then it uses this information to calculate the distance traveled. Each subproblem is solved in sequence, with the solution to the first subproblem feeding into the second. The solutions are then aggregated to arrive at the final answer. This method emphasizes the key principles of problem decomposition and sequential problem solving, enabling the model to manage and solve complex tasks more effectively.

![Refer to caption](x2.png) Figure 10: Illustration of Least-to-Most Prompting Applied to a Mathematical Word.

Upon rigorous experimentation in domains including symbolic manipulation, compositional generalization, and mathematical reasoning, [[74](https://arxiv.org/html/2310.14735v5#bib.bib74)] substantiate that the least-to-most prompting paradigm exhibits the capacity to generalize across challenges of greater complexity than those initially presented in the prompts. [[75](https://arxiv.org/html/2310.14735v5#bib.bib75)] introduced Program Aided Language models (PAL), using the LLMs to read natural language problems and generate programs as the intermediate reasoning steps. By using least to most prompting, PAL shows enhancement on GSM8K [[60](https://arxiv.org/html/2310.14735v5#bib.bib60)] and SVAMP [[76](https://arxiv.org/html/2310.14735v5#bib.bib76)], which are benchmarks about complex mathematical problems for LLMs.

###  3.5 Tree of thoughts

The “tree of thoughts” (ToT) prompting technique in LLMs is an advanced method that employs a structured approach to guide LLMs in their reasoning and response generation processes. It enhances problem-solving by exploring multiple reasoning paths, termed ‘thoughts’. Unlike traditional linear prompts, ToT allows LLMs to consider various possible solutions and strategies, including looking ahead, backtracking, and self-evaluation, making it more interactive and adaptable to the complexity of the task at hand. This approach fosters more dynamic and deliberate decision-making in complex problem-solving tasks, moving beyond a rigid, hierarchical prompt structure to a more flexible and adaptive reasoning process [[77](https://arxiv.org/html/2310.14735v5#bib.bib77)]. The ToT approach, for instance, when applied to complex mathematical problem-solving, prompts the model to generate various potential solutions and evaluate them, rather than simply asking for a solution.

[[77](https://arxiv.org/html/2310.14735v5#bib.bib77)] demonstrates that this formulation is more versatile and can handle challenging tasks where standard prompts might fall short. Another research by [[78](https://arxiv.org/html/2310.14735v5#bib.bib78)] further emphasizes the potential of this technique in enhancing the performance of LLMs by structuring their thought processes.

[[7](https://arxiv.org/html/2310.14735v5#bib.bib7)] introduces the “tree-of-thought prompting”, an approach that assimilates the foundational principles of the ToT frameworks and transforms them into a streamlined prompting methodology. This technique enables LLMs to assess intermediate cognitive constructs within a singular prompt. An exemplar ToT prompt is delineated in Figure [11](https://arxiv.org/html/2310.14735v5#S3.F11 "Figure 11 ‣ 3.5 Tree of thoughts ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review").

Imagine three different experts answering this question.   
All experts will write down 1 step of their thinking,   
then share it with the group.   
Then all experts will go on to the next step, etc.   
If any expert realizes they’re wrong at any point then they leave.   
The question is…

Tree of thoughts prompting

Figure 11: A sample ToT prompt [[7](https://arxiv.org/html/2310.14735v5#bib.bib7)].

###  3.6 Graph of thoughts

Unlike the “chain-of-thoughts” or “tree of thoughts” paradigms, the “graph of thoughts” (GoT) framework [[79](https://arxiv.org/html/2310.14735v5#bib.bib79)] offers a more intricate method of representing the information generated by LLMs. The core concept behind GoT is to model this information as an arbitrary graph. In this graph, individual units of information, termed “LLM thoughts”, are represented as vertices. The edges of the graph, on the other hand, depict the dependencies between these vertices. This unique representation allows for the combination of arbitrary LLM thoughts, thereby creating a synergistic effect in the model’s outputs.

In the context of addressing intricate challenges, LLMs utilizing the GoT framework might initially produce several autonomous thoughts or solutions. These individual insights can subsequently be interlinked based on their pertinence and interdependencies, culminating in a detailed graph. This constructed graph permits diverse traversal methods, ensuring the final solution is both precise and comprehensive, encompassing various dimensions of the challenge.

The efficacy of the GoT framework is anchored in its adaptability and the profound insights it can yield, particularly for intricate issues necessitating multifaceted resolutions. Nonetheless, it is imperative to recognize that while GoT facilitates a systematic approach to problem-solving, it also necessitates a profound comprehension of the subject matter and meticulous prompt design to realize optimal outcomes [[80](https://arxiv.org/html/2310.14735v5#bib.bib80)].

###  3.7 Decomposed prompting

Decomposed Prompting (DECOMP) [[81](https://arxiv.org/html/2310.14735v5#bib.bib81)] is a modular approach designed to tackle complex tasks by breaking them down into simpler, manageable sub-tasks. This methodology leverages the capabilities of LLMs by creating a systematic process where each sub-task is handled by specialized handlers. The approach not only simplifies the problem-solving process but also enhances the flexibility and efficiency of task handling.

![Refer to caption](x3.png) Figure 12: Key components of DECOMP

Four key components of this method are shown in Figure [12](https://arxiv.org/html/2310.14735v5#S3.F12 "Figure 12 ‣ 3.7 Decomposed prompting ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"). The core of DECOMP involves a decomposer LLM that generates a prompting program P𝑃Pitalic_P for a complex task Q𝑄Qitalic_Q. The program P𝑃Pitalic_P is a sequence of steps, each step directing a simpler sub-query to a function within an auxiliary set of sub-task functions F𝐹Fitalic_F. The program can be represented as:

| P={(f1,Q1,A1),…,(fk,Qk,Ak)}𝑃subscript𝑓1subscript𝑄1subscript𝐴1…subscript𝑓𝑘subscript𝑄𝑘subscript𝐴𝑘P=\\{(f_{1},Q_{1},A_{1}),\ldots,(f_{k},Q_{k},A_{k})\\}italic_P = { ( italic_f start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , italic_Q start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , italic_A start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT ) , … , ( italic_f start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT , italic_Q start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT , italic_A start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT ) } |   
---|---|---  
  
where Aksubscript𝐴𝑘A_{k}italic_A start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT is the final answer predicted by P𝑃Pitalic_P, and Qisubscript𝑄𝑖Q_{i}italic_Q start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT is a sub-query directed to the sub-task function fi∈Fsubscript𝑓𝑖𝐹f_{i}\in Fitalic_f start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ∈ italic_F. A high-level imperative controller manages the execution of P𝑃Pitalic_P, passing inputs and outputs between the decomposer and sub-task handlers until the final output is obtained.

To teach the decomposer LLM, in-context examples are used. These examples demonstrate the decomposition of complex queries into simpler sub-queries. Each example Ejsubscript𝐸𝑗E_{j}italic_E start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT takes the form:

| Ej=(Qj,{(fj,1,Qj,1,Aj,1),…,(fj,kj,Qj,kj,Aj,kj)})subscript𝐸𝑗subscript𝑄𝑗subscript𝑓𝑗1subscript𝑄𝑗1subscript𝐴𝑗1…subscript𝑓𝑗subscript𝑘𝑗subscript𝑄𝑗subscript𝑘𝑗subscript𝐴𝑗subscript𝑘𝑗E_{j}=\left(Q_{j},\left\\{(f_{j,1},Q_{j,1},A_{j,1}),\ldots,(f_{j,k_{j}},Q_{j,k_% {j}},A_{j,k_{j}})\right\\}\right)italic_E start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT = ( italic_Q start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT , { ( italic_f start_POSTSUBSCRIPT italic_j , 1 end_POSTSUBSCRIPT , italic_Q start_POSTSUBSCRIPT italic_j , 1 end_POSTSUBSCRIPT , italic_A start_POSTSUBSCRIPT italic_j , 1 end_POSTSUBSCRIPT ) , … , ( italic_f start_POSTSUBSCRIPT italic_j , italic_k start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT end_POSTSUBSCRIPT , italic_Q start_POSTSUBSCRIPT italic_j , italic_k start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT end_POSTSUBSCRIPT , italic_A start_POSTSUBSCRIPT italic_j , italic_k start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT end_POSTSUBSCRIPT ) } ) |   
---|---|---  
  
where Aj,kj=Ajsubscript𝐴𝑗subscript𝑘𝑗subscript𝐴𝑗A_{j,k_{j}}=A_{j}italic_A start_POSTSUBSCRIPT italic_j , italic_k start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT end_POSTSUBSCRIPT = italic_A start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT is the final answer for Qjsubscript𝑄𝑗Q_{j}italic_Q start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT, and (Qj,1,…,Qj,kj)subscript𝑄𝑗1…subscript𝑄𝑗subscript𝑘𝑗(Q_{j,1},\ldots,Q_{j,k_{j}})( italic_Q start_POSTSUBSCRIPT italic_j , 1 end_POSTSUBSCRIPT , … , italic_Q start_POSTSUBSCRIPT italic_j , italic_k start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT end_POSTSUBSCRIPT ) represents the decomposition of Qjsubscript𝑄𝑗Q_{j}italic_Q start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT. Each sub-task function f𝑓fitalic_f is operationalized through sub-task handlers, which can be additional LLM prompts or symbolic or learned functions [[81](https://arxiv.org/html/2310.14735v5#bib.bib81)]. An illustration of the process flow is shown in Figure [13](https://arxiv.org/html/2310.14735v5#S3.F13 "Figure 13 ‣ 3.7 Decomposed prompting ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review").

![Refer to caption](x4.png) Figure 13: An example of the process flow of DECOMP.

The DECOMP approach has several advantages. First, its modularity allows each sub-task handler to be independently optimized, debugged, and upgraded, which facilitates systematic performance improvements and easier integration of new methods or models. Second, DECOMP can incorporate error-correcting sub-task handlers, improving the overall accuracy and reliability of the system. Third, the approach allows for diverse decomposition structures, including hierarchical and recursive decompositions, which are particularly useful for handling complex and large-scale problems. Finally, sub-task handlers can be shared across different tasks, enhancing the efficiency of the problem-solving process.

DECOMP and Least-to-Most Prompting [[74](https://arxiv.org/html/2310.14735v5#bib.bib74)] both decompose complex tasks to enhance large language models’ reasoning abilities, but DECOMP distinguishes itself through its flexible, modular approach. Unlike Least-to-Most Prompting’s linear progression from easy to hard sub-questions, DECOMP allows for non-linear and recursive decomposition, with dedicated sub-task handlers that can be independently optimized and replaced. This modularity not only enhances flexibility and reusability across tasks but also introduces potential error-correcting mechanisms, making DECOMP more robust and adaptable to complex, multi-step reasoning tasks. While DECOMP has demonstrated superior performance in specific domains, such as symbolic reasoning and multi-step question answering, its advantages over Least-to-Most Prompting may vary depending on the nature of the task [[81](https://arxiv.org/html/2310.14735v5#bib.bib81)].

In case studies, DECOMP demonstrated superior performance in various scenarios. For instance, in the k-th letter concatenation task, DECOMP outperformed CoT prompting by effectively teaching the sub-task of extracting the k-th letter through further decomposition. In list reversal, DECOMP showed better length generalization compared to CoT by recursively decomposing the task into reversing smaller sub-lists, achieving higher accuracy for longer input sequences. In long-context question answering (QA), DECOMP allowed for handling more examples than feasible with CoT prompting, leading to improved performance. In open-domain QA, incorporating symbolic retrieval APIs within the DECOMP framework enhanced performance on multi-hop QA datasets compared to CoT prompting. Additionally, in Math QA, DECOMP improved accuracy by post-processing CoT prompts to fix frequent formatting errors, resulting in significant performance gains [[81](https://arxiv.org/html/2310.14735v5#bib.bib81)].

By leveraging the modular, flexible, and systematic approach of DECOMP, complex tasks can be effectively decomposed and solved, showcasing its superiority over traditional CoT prompting and other contemporary methods.

###  3.8 Active prompt

The adoption of the active prompt [[82](https://arxiv.org/html/2310.14735v5#bib.bib82)] method marks a significant advancement in the utilization of LLMs for complex reasoning tasks. The active prompt method does not involve the traditional process of prefix-tuning [[83](https://arxiv.org/html/2310.14735v5#bib.bib83)]. Instead, it focuses on improving the reasoning capabilities of LLMs through strategic selection and annotation of task-specific examples. By systematically selecting and annotating the most uncertain questions, this method not only refines the model’s understanding but also leverages human expertise more effectively [[84](https://arxiv.org/html/2310.14735v5#bib.bib84)]. The process begins with the generation of multiple predictions for each question, followed by the calculation of uncertainty (uncertainty estimation) [[85](https://arxiv.org/html/2310.14735v5#bib.bib85), [86](https://arxiv.org/html/2310.14735v5#bib.bib86)] using various metrics such as disagreement, entropy, and variance. This strategic selection process ensures that the most informative questions are prioritized for annotation. The human annotation phase is crucial, as it involves providing detailed chain-of-thought reasoning and answers, which are then used to prompt the LLM during inference. This annotated data serves as exemplars, guiding the model through complex reasoning pathways and enhancing its predictive accuracy. The application of self-consistency [[24](https://arxiv.org/html/2310.14735v5#bib.bib24)] techniques further solidifies the model’s reliability by selecting the most consistent answers from multiple reasoning paths. The key innovation of this method is the thought of finding out the most efficient one-shot or few-shot [[43](https://arxiv.org/html/2310.14735v5#bib.bib43)] examples, so it improves the inference ability of specific fields. A concrete process illustration is shown in Figure [14](https://arxiv.org/html/2310.14735v5#S3.F14 "Figure 14 ‣ 3.8 Active prompt ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review").

![Refer to caption](x5.png) Figure 14: Illustration of the whole process. (1) Uncertainty Estimation. (2) Collection, Ranking and Selection. (3) Annotation (by human). (4) Inference.

The active prompt method offers several key benefits, including efficient task adaptation and significant performance improvements across various reasoning domains. This approach aligns with the broader trend towards more interactive and adaptive AI systems, emphasizing the importance of responsive design in prompt engineering. Its ability to reduce human engineering efforts by focusing on the most uncertain and informative questions makes it an important tool for advancing LLM capabilities. This method not only enhances the quality of task-specific prompts but also maximizes the use of human expertise, paving the way for more sophisticated and accurate AI systems [[82](https://arxiv.org/html/2310.14735v5#bib.bib82)].

###  3.9 Prompt pattern catalog

A Prompt Pattern Catalog [[87](https://arxiv.org/html/2310.14735v5#bib.bib87)] is an organized collection of prompt templates and patterns designed to enhance the effectiveness of prompt engineering, particularly for LLMs such as ChatGPT. This methodology involves creating a standardized set of prompt patterns that can be applied across various tasks, ensuring consistency and optimizing the performance of models through systematic prompt design. By developing a catalog of prompt patterns, researchers and practitioners can ensure a consistent approach to prompt engineering, reducing variability and errors from ad hoc prompt creation [[87](https://arxiv.org/html/2310.14735v5#bib.bib87), [88](https://arxiv.org/html/2310.14735v5#bib.bib88)]. Predefined prompt patterns streamline the process of prompt engineering, saving time and resources by allowing practitioners to select and adapt patterns rather than crafting new prompts from scratch. A well-designed prompt pattern catalog includes patterns for various contexts and applications, enabling models to be quickly adapted to new tasks and domains by selecting the most appropriate patterns. Systematic use of optimized prompt patterns enhances model performance by providing more effective and contextually appropriate prompts, leading to better task-specific results [[89](https://arxiv.org/html/2310.14735v5#bib.bib89)].

The central methodology of this research involves the conceptualization and application of prompt patterns, which are reusable solutions to common problems encountered when interacting with LLMs. These prompt patterns are analogous to design patterns in software engineering, providing structured and documented approaches to enhance the output and interaction quality of LLMs. The framework for documenting these prompt patterns includes a detailed structure that ensures their adaptability across different domains [[87](https://arxiv.org/html/2310.14735v5#bib.bib87)].

To systematically categorize these prompt patterns, the authors have divided them into five primary categories: Input Semantics, Output Customization, Error Identification, Prompt Improvement, and Interaction. This classification helps in organizing the patterns based on their functional roles and the specific problems they address. Within this framework, the research introduces a comprehensive catalog of 16 distinct prompt patterns. Each pattern is meticulously documented with the following components: name and classification, intent and context, motivation, structure and key ideas, example implementation, and practical consequences. The prompt patterns cover a wide range of functionalities. For instance, the Input Semantics category includes patterns such as Meta Language Creation, which helps in defining custom input languages for LLMs. The Output Customization category features patterns such as Output Automater and Visualization Generator, which tailor the generated outputs to specific formats or visualizations. Error Identification patterns such as Fact Check List ensure the accuracy of generated content by highlighting critical facts for verification. Prompt Improvement patterns, including Question Refinement and Alternative Approaches, enhance the quality of interactions by refining questions and suggesting multiple ways to achieve a goal. Lastly, Interaction patterns such as Flipped Interaction and Game Play facilitate dynamic and engaging user-LLM interactions [[87](https://arxiv.org/html/2310.14735v5#bib.bib87)].

The methodology also emphasizes the combinatory use of these patterns to tackle more complex prompt engineering tasks. By providing detailed examples and practical implementations, the research demonstrates how multiple prompt patterns can be integrated to create sophisticated and efficient prompting strategies. This structured approach not only improves the effectiveness of LLMs in various applications but also contributes to the broader understanding and advancement of prompt engineering as a field [[87](https://arxiv.org/html/2310.14735v5#bib.bib87)].

Research supports the effectiveness of prompt pattern catalogs. [[87](https://arxiv.org/html/2310.14735v5#bib.bib87)] outlines that the development and use of a prompt pattern catalog can improve the effectiveness and efficiency of prompt engineering with LLMs. [[89](https://arxiv.org/html/2310.14735v5#bib.bib89)] explores how predefined structured prompt patterns can enhance user interaction and improve model outputs in conversational AI. [[88](https://arxiv.org/html/2310.14735v5#bib.bib88)] investigates the application of prompt engineering patterns in enterprise settings, demonstrating their utility in optimizing model performance across various tasks. Additionally, [[90](https://arxiv.org/html/2310.14735v5#bib.bib90)] highlights the benefits of using predefined structured prompt patterns in software development, demonstrating significant improvements in code quality, requirements elicitation, and refactoring efficiency.

###  3.10 Prompt optimization

In the domain of prompt engineering for LLMs, the challenge of crafting effective prompts remains a significant barrier due to the extensive manual effort and expertise required. Prompt optimization is a critical technique for improving the performance of LLMs by refining the input prompts that guide their responses. The process of prompt optimization systematically adjusts these prompts to enhance accuracy and relevance, reducing the need for manual trial and error.

Several methods have been developed to automate prompt optimization, including gradient-based approaches such as Prompt Optimization with Textual Gradients (ProTeGi) [[91](https://arxiv.org/html/2310.14735v5#bib.bib91)], which uses text-based gradients to iteratively refine prompts, and black-box methods that optimize prompts based solely on output performance without requiring model internals. Additionally, model-adaptive techniques, such as Model-Adaptive Prompt Optimization (MAPO) [[92](https://arxiv.org/html/2310.14735v5#bib.bib92)], tailor the optimization to the specific characteristics of the LLM, potentially offering superior results. Each method has its advantages: gradient-based techniques are efficient and directed, black-box approaches are broadly applicable and easy to implement, and model-adaptive methods provide customized optimization for specific models. The choice of method depends on task requirements, model complexity, and available resources.

####  3.10.1 Prompt optimization with textual gradients

Prompt Optimization with Textual Gradients (ProTeGi) [[91](https://arxiv.org/html/2310.14735v5#bib.bib91)] is inspired by gradient descent, a fundamental technique in optimization, but adapts this concept to the discrete and non-parametric nature of natural language processing. Instead of relying on numerical gradients, ProTeGi generates “textual gradients”, which are natural language descriptions of the flaws in a given prompt based on its performance on a small batch of data. These gradients indicate the semantic direction in which the prompt needs to be improved.

ProTeGi further enhances this optimization process by applying these textual gradients to modify the prompt in the opposite semantic direction, akin to a reverse gradient descent in the language space. This iterative process is guided by a beam search algorithm combined with a bandit selection strategy, which efficiently explores the space of possible prompts and selects the most promising candidates for further refinement [[91](https://arxiv.org/html/2310.14735v5#bib.bib91)].

The effectiveness of ProTeGi has been demonstrated across multiple NLP tasks, including sentiment analysis, fake news detection, and the novel problem of LLM jailbreak detection. Experimental results indicate that ProTeGi can significantly improve prompt performance, with reported gains of up to 31% over initial prompts, while also surpassing existing prompt optimization methods in efficiency and accuracy. This method provides a robust, data-driven approach to prompt engineering, offering a scalable solution that can adapt to various tasks without requiring access to the internal states of LLMs [[91](https://arxiv.org/html/2310.14735v5#bib.bib91)].

####  3.10.2 Black-box prompt optimization

In recent prompt engineering research, the challenge of aligning LLMs with human intent without model retraining has garnered significant attention. Traditional alignment techniques, such as Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO), typically require substantial computational resources and direct access to model parameters, which are not always feasible or efficient, particularly with closed-source models such as GPT-4 or Claude-2. In response to these limitations, a novel method called Black-box Prompt Optimization (BPO) [[93](https://arxiv.org/html/2310.14735v5#bib.bib93)] has been introduced, providing a promising alternative for enhancing model alignment through prompt optimization alone.

BPO shifts the focus from model-centric to input-centric optimization, where the key idea is to refine the user’s prompts rather than altering the model’s internal parameters. This approach leverages feedback from pre-existing datasets that contain human preferences, creating pairs of original and optimized prompts. These pairs are then used to train a sequence-to-sequence model designed to rewrite prompts in a way that improves the alignment of LLM outputs with human expectations [[93](https://arxiv.org/html/2310.14735v5#bib.bib93)].

The BPO method offers several advantages. First, it is model-agnostic, allowing it to be applied across various LLMs, whether open-source or API-based, without requiring access to the model’s internals. Second, it enhances interpretability, as the changes made to prompts are transparent and directly observable, providing clear insights into how and why a particular prompt leads to better alignment. Third, empirical results demonstrate that BPO not only improves the alignment of models such as GPT-3.5 and LLaMA-2 but also outperforms RLHF and DPO when used independently or in conjunction with these methods [[93](https://arxiv.org/html/2310.14735v5#bib.bib93)].

####  3.10.3 Model-adaptive prompt optimization

Traditionally, prompt optimization has focused on tailoring prompts to specific tasks to enhance model performance. However, [[92](https://arxiv.org/html/2310.14735v5#bib.bib92)] highlighted the necessity of adapting prompts not just to tasks but also to the specific characteristics of different LLMs. This shift in perspective has led to the development of Model-Adaptive Prompt Optimization (MAPO), a novel approach designed to fine-tune prompts for individual LLMs, thereby maximizing their effectiveness across various downstream tasks. MAPO addresses the inherent variability in how different LLMs respond to the same prompt by introducing a two-phase optimization process. The first phase involves establishing a warm-up dataset, where candidate prompts are generated and evaluated for their suitability to each LLMs. This is followed by a combination of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), particularly employing techniques such as Proximal Policy Optimization (PPO) and Ranking Responses from Model Feedback (RRMF). This joint learning approach refines the prompts, ensuring they align with the specific preferences of each LLMs.

Empirical studies demonstrate that MAPO significantly improves performance in tasks such as question-answering, classification, and text generation when compared to conventional task-specific prompt optimization methods. By shifting the focus from a one-size-fits-all approach to a more nuanced, model-specific strategy, MAPO enhances the robustness and generalization of LLMs, making it a powerful tool in the prompt engineering toolkit [[92](https://arxiv.org/html/2310.14735v5#bib.bib92)].

####  3.10.4 PromptAgent

The PromptAgent method suggests framing prompt optimization as a strategic planning problem. A kind core of this method is the use of Monte Carlo Tree Search (MCTS), a principled planning algorithm that strategically navigates the vast space of expert-level prompts. Unlike conventional methods that generate prompts through local variations, PromptAgent employs a trial-and-error mechanism, inspired by human problem-solving strategies. This approach allows the model to iteratively refine prompts based on error feedback, simulating future rewards and prioritizing high-reward paths [[94](https://arxiv.org/html/2310.14735v5#bib.bib94)]. Another core of this method, for instance, PromptSource [[95](https://arxiv.org/html/2310.14735v5#bib.bib95)], collects over 2,000 open-source prompts for roughly 170 datasets, by dataset exploration, prompt writing and documentation to provide an enhanced prompt.

PromptAgent’s effectiveness has been demonstrated across a diverse set of tasks, spanning general NLP challenges and domain-specific applications such as biomedical text processing. By autonomously generating prompts that incorporate domain-specific knowledge and detailed task instructions, PromptAgent consistently outperforms both human-designed prompts and other automated optimization methods [[94](https://arxiv.org/html/2310.14735v5#bib.bib94)], highlighting the importance of integrating strategic planning and self-reflection capabilities into prompt optimization frameworks.

####  3.10.5 Reinforcement learning

Reinforcement Learning (RL) for prompt optimization is an advanced technique designed to enhance the performance of LLMs by iteratively refining the prompts used during training and inference. This method utilizes the principles of reinforcement learning to navigate the complex parameter space of large models, optimizing the prompts for improved task-specific performance. In RL for prompt optimization, a reward function is defined to evaluate the effectiveness of different prompts based on the model’s output. The model then uses this feedback to adjust and optimize the prompts through a series of iterations, ensuring that the prompts evolve to maximize performance on the target task by leveraging the model’s ability to learn from its interactions with the environment [[96](https://arxiv.org/html/2310.14735v5#bib.bib96)].

Consider the task of VQA, where the goal is to generate accurate answers to questions based on visual input. Using RL for prompt optimization, the model can start with a set of initial prompts and iteratively refine them based on the accuracy of the generated answers. For instance, if the model is asked, “What is the color of the car in the image?” the initial prompts might produce varied responses. The reward function will assess these responses, favoring prompts that lead to correct answers. Over multiple iterations, the model learns to generate more precise prompts, improving its ability to accurately answer similar questions in the future [[97](https://arxiv.org/html/2310.14735v5#bib.bib97)].

####  3.10.6 GPTs (plugins)

Before ending this discussion on prompt optimization techniques, we need to mention the use of external prompt engineering assistants that have been developed recently and exhibit promising potential. Unlike the methods introduced previously, these instruments can help us to polish the prompt directly. They are adept at analyzing user inputs and subsequently producing pertinent outputs within a context that is defined by itself, thereby amplifying the efficacy of prompts. Some of the plugins provided by the OpenAI GPT store are good examples of such tools [[98](https://arxiv.org/html/2310.14735v5#bib.bib98)]. Some popular GPT store apps that specialize in generating or optimizing prompts are shown in Figure [15](https://arxiv.org/html/2310.14735v5#S3.F15 "Figure 15 ‣ 3.10.6 GPTs \(plugins\) ‣ 3.10 Prompt optimization ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review").

![Refer to caption](x6.png) Figure 15: Examples of GPT apps that specialize in generating or optimizing prompts [[99](https://arxiv.org/html/2310.14735v5#bib.bib99)].

In certain implementations, the definition of a plugin is incorporated into the prompt, altering the output [[100](https://arxiv.org/html/2310.14735v5#bib.bib100)]. Such integration may impact the manner in which LLMs interpret and react to the prompts, illustrating a connection between prompt engineering and plugins. Plugins mitigate the laborious nature of intricate prompt engineering, enabling the model to more proficiently comprehend or address user inquiries without necessitating excessively detailed prompts. Consequently, plugins can bolster the efficacy of prompt engineering while promoting enhanced user-centric efficiency. These tools, akin to packages, can be seamlessly integrated into Python and invoked directly [[101](https://arxiv.org/html/2310.14735v5#bib.bib101), [102](https://arxiv.org/html/2310.14735v5#bib.bib102)]. For instance, the “Prompt Enhancer” pluging [[103](https://arxiv.org/html/2310.14735v5#bib.bib103)], developed by AISEO [[104](https://arxiv.org/html/2310.14735v5#bib.bib104)], can be invoked by starting the prompt with the word “AISEO” to let the AISEO prompt generator automatically enhance the LLM prompt provided. Similarly, another plugin called “Prompt Perfect”, can be used by starting the prompt with ‘perfect’ to automatically enhance the prompt, aiming for the “perfect” prompt for the task at hand [[105](https://arxiv.org/html/2310.14735v5#bib.bib105), [106](https://arxiv.org/html/2310.14735v5#bib.bib106)]. Nevertheless, while the use of plugins to enhance prompts is simple and handy, it is not always clear which prompt engineering technique, or combination of techniques, is implemented by a given plugin, given the closed-source nature of most plugins.

###  3.11 Retrieval augmentation

Another direction of prompt engineering research is to aim to reduce hallucinations. When using AIGC tools such as GPT-4, it is common to face a problem called “hallucinations”, which refer to the presence of unreal or inaccurate information in the model’s generated output [[107](https://arxiv.org/html/2310.14735v5#bib.bib107), [26](https://arxiv.org/html/2310.14735v5#bib.bib26)]. While these outputs may be grammatically correct, they can be inconsistent with facts or lack real-world data support. Hallucinations arise because the model may not have found sufficient evidence in its training data to support its responses, or it may overly generalize certain patterns when attempting to generate fluent and coherent output [[108](https://arxiv.org/html/2310.14735v5#bib.bib108)].

An approach to reduce hallucinations and enhance the effectiveness of prompts is the so-called retrieval augmentation technique, which aims at incorporating up-to-date external knowledge into the model’s input [[109](https://arxiv.org/html/2310.14735v5#bib.bib109), [110](https://arxiv.org/html/2310.14735v5#bib.bib110)]. It is emerging as an AI framework for retrieving facts from external sources. [[111](https://arxiv.org/html/2310.14735v5#bib.bib111)] examines the augmentation of context retrieval through the incorporation of external information. It proposes a sophisticated operation: the direct concatenation of pertinent information obtained from an external source to the prompt, which is subsequently treated as foundational knowledge for input into the expansive language model. Additionally, the paper introduces auto-regressive techniques for both retrieval and decoding, facilitating a more nuanced approach to information retrieval and fusion. [[111](https://arxiv.org/html/2310.14735v5#bib.bib111)] demonstrates that in-context retrieval-augmented language models, when constructed upon readily available general-purpose retrievers, yield significant LLM enhancements across a variety of model dimensions and diverse corpora. In another research, [[112](https://arxiv.org/html/2310.14735v5#bib.bib112)] showed that GPT-3 can reduce hallucinations by studying various implementations of the retrieval augmentation concept, such as Retrieval Augmented Generation (RAG) [[113](https://arxiv.org/html/2310.14735v5#bib.bib113)], Fusion-in-Decoder (FiD) [[114](https://arxiv.org/html/2310.14735v5#bib.bib114)], Seq2seq [[115](https://arxiv.org/html/2310.14735v5#bib.bib115), [116](https://arxiv.org/html/2310.14735v5#bib.bib116), [117](https://arxiv.org/html/2310.14735v5#bib.bib117)] and others. [[118](https://arxiv.org/html/2310.14735v5#bib.bib118)] developed the Chain-of-Verification (CoVe) approach to reduce hallucinations, based on letting the LLM deliberate on its own responses before self-correcting them. They suspect that extending this approach with retrieval augmentation would likely bring further gains. UNIWEB [[119](https://arxiv.org/html/2310.14735v5#bib.bib119)] converting knowledge-intensive tasks into a unified text-to-text framework and treating the web as a general source of knowledge.

###  3.12 Reasoning and active interaction

This subsection explores two advanced techniques that enhance the capabilities of LLMs by integrating reasoning with interaction through external tools or other action abilities. Automatic Reasoning and Tool Usage (ART) combines CoT prompting with the use of specialized tools. By guiding LLMs through multi-step reasoning and incorporating resources such as calculators and databases, ART improves the logical coherence and accuracy of model outputs. The ReAct Framework (Reasoning and Acting) synergizes reasoning with actionable steps. It prompts LLMs to devise logical sequences and interact dynamically with external tools, enabling them to handle complex, multi-step tasks efficiently. Both ART and ReAct represent significant advancements in prompt engineering, enhancing the range and reliability of tasks that LLMs can perform through the integration of reasoning and interaction.

####  3.12.1 Automatic reasoning and tool usage

ART is an advanced prompting technique that combines the principles of automatic CoT prompting with the strategic utilization of external tools. This method aims to enhance the reasoning capabilities of LLMs by guiding them through multi-step reasoning processes and leveraging specialized tools to achieve more accurate and relevant outputs [[120](https://arxiv.org/html/2310.14735v5#bib.bib120)].

ART builds on the CoT prompting technique, which encourages models to generate intermediate reasoning steps before arriving at a final answer. In ART, these reasoning steps are augmented by incorporating external tools such as calculators, databases, or other software applications. The integration of tools helps LLMs to perform tasks that require precise calculations, access to updated information, or specialized data processing that the model alone may not handle effectively.

For example, a prompt designed using ART might guide an LLM to first outline the steps required to solve a complex mathematical problem and then use a calculator tool to perform the necessary calculations. This combination of reasoning and tool usage ensures that the model’s outputs are both logically coherent and computationally accurate.

[[121](https://arxiv.org/html/2310.14735v5#bib.bib121)] have demonstrated that ART can help models navigate complex problem spaces more effectively by breaking down tasks into manageable steps and utilizing appropriate tools at each stage. For instance, the integration of ART in natural language processing tasks has shown promising results in areas such as automated customer service, where models need to access and process information dynamically [[122](https://arxiv.org/html/2310.14735v5#bib.bib122)].

Moreover, ART’s approach aligns with ongoing efforts to develop more robust and versatile AI systems capable of handling real-world tasks that demand a combination of cognitive and computational skills. [[123](https://arxiv.org/html/2310.14735v5#bib.bib123)] explores advanced ART techniques to achieve better accuracy and reliability in AI applications. These findings underscore the importance of ART in enhancing the functionality and performance of LLMs, making them more adept at handling a broader range of tasks, in particular technical problem-solving tasks that require specific and precise outputs such as financial calculations or data analysis.

####  3.12.2 ReAct framework

The ReAct Framework, which stands for Reasoning and Acting, synergizes the processes of reasoning and action to enable LLMs to not only think through problems but also interact with external tools and environments to achieve more accurate and contextually appropriate outcomes.

The ReAct Framework operates by prompting LLMs to generate both reasoning traces and task-specific actions. This dual approach ensures that the model first contemplates the problem, devises a logical sequence of thoughts, and then executes actions that may involve querying external databases, using calculators, or interacting with other software tools. This method is particularly effective in scenarios requiring detailed reasoning followed by specific actions, thus ensuring the LLM can handle complex, multi-step tasks efficiently [[124](https://arxiv.org/html/2310.14735v5#bib.bib124)].

For example, in a task involving financial analysis, the ReAct framework would first prompt the LLM to outline the necessary steps to evaluate a portfolio. Subsequently, the model could use financial analysis tools to gather current market data and perform calculations, integrating these results into the final analysis. This combination of reasoning and action leads to more robust and reliable outcomes compared to using static prompts alone. Another concrete example is shown in Figure [16](https://arxiv.org/html/2310.14735v5#S3.F16 "Figure 16 ‣ 3.12.2 ReAct framework ‣ 3.12 Reasoning and active interaction ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review").

![Refer to caption](x7.png) Figure 16: An example of ReAct method.

By integrating reasoning and action, ReAct enables LLMs to make more informed and accurate decisions, in particular in fields such as finance, healthcare, and legal analysis, where decisions must be based on comprehensive data and logical reasoning [[125](https://arxiv.org/html/2310.14735v5#bib.bib125)]. Additionally, the framework’s ability to interact with external tools ensures that the information used in decision-making is up-to-date and relevant, reducing the likelihood of errors due to outdated or incomplete data [[126](https://arxiv.org/html/2310.14735v5#bib.bib126)].

Implementing the ReAct framework is not a trivial task, as it involves developing prompts that guide LLMs through both thought processes and actions. This requires a detailed understanding of the task at hand and the tools available, ensuring that the model can seamlessly transition from reasoning to action.

##  4 Methodologies for multimodal large models

In recent years, vision-language models (VLMs) have made significant advancements in multimodal learning by combining visual and linguistic information. These models have demonstrated strong capabilities in tasks such as image description and visual question answering (VQA) [[127](https://arxiv.org/html/2310.14735v5#bib.bib127), [128](https://arxiv.org/html/2310.14735v5#bib.bib128), [129](https://arxiv.org/html/2310.14735v5#bib.bib129), [130](https://arxiv.org/html/2310.14735v5#bib.bib130)]. While this review primarily focuses on the potential of prompt engineering in LLMs, it is also pertinent to briefly introduce the importance of VLMs and their applications in multimodal tasks to provide a more comprehensive perspective.

VLMs are based on the transformer architecture, and are trained on extensive datasets to learn complex semantic relationships. However, unlike early unimodal models, VLMs process both textual and visual information, enabling them to establish connections between image understanding and text generation. As can be expected, this multimodal integration makes VLMs particularly effective at handling complex tasks that involve both images and text.

To seamlessly integrate and interpret these diverse data types, VLMs require sophisticated prompt designs that ensure contextual coherence and accuracy [[131](https://arxiv.org/html/2310.14735v5#bib.bib131), [132](https://arxiv.org/html/2310.14735v5#bib.bib132)]. Challenges such as data alignment, modality integration, and context preservation are addressed through advanced techniques such as Context Optimization (CoOp, subsection [4.3](https://arxiv.org/html/2310.14735v5#S4.SS3 "4.3 Context optimization ‣ 4 Methodologies for multimodal large models ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")) and Multimodal Prompt Learning (MaPLe, subsection [4.5](https://arxiv.org/html/2310.14735v5#S4.SS5 "4.5 Multimodal prompt learning ‣ 4 Methodologies for multimodal large models ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")). These advanced prompt engineering techniques enhance the ability of VLMs to generate nuanced and contextually rich outputs, thereby facilitating their effective utilization in various applications and enabling them to tackle more complex tasks [[131](https://arxiv.org/html/2310.14735v5#bib.bib131)].

###  4.1 Zero-shot and few-shot prompting

Zero-shot and few-shot prompting, which have already been discussed in subsection [2.7](https://arxiv.org/html/2310.14735v5#S2.SS7 "2.7 One-shot or few-shot prompting ‣ 2 Basics of prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") in the context of LLMs, are also pivotal techniques in the realm of VLMs, enabling these models to handle tasks with minimal or no task-specific training data. Zero-shot prompting allows models to perform tasks without any specific examples provided during training, relying entirely on their pre-trained knowledge to generalize across new tasks and domains. For example, a model such as CLIP can be prompted with a textual description to classify images into categories it has never explicitly been trained on [[3](https://arxiv.org/html/2310.14735v5#bib.bib3)]. On the other hand, few-shot prompting involves providing the model with a small number of examples during inference, significantly enhancing the model’s ability to generalize with limited data [[16](https://arxiv.org/html/2310.14735v5#bib.bib16)].

In relation to these methods, [[97](https://arxiv.org/html/2310.14735v5#bib.bib97)] systematically explored a range of prompting techniques for zero-shot and few-shot visual question answering (VQA) in vision-language models (VLMs), highlighting the impact of question templates, the integration of image captions, and the application of chain-of-thought reasoning on model performance. [[16](https://arxiv.org/html/2310.14735v5#bib.bib16)] showed the application of these techniques in CLIP, highlighting the model’s ability to generalize across different domains. Additionally, [[133](https://arxiv.org/html/2310.14735v5#bib.bib133)] presented a method for adapting CLIP to few-shot classification tasks without additional training, emphasizing practical benefits in real-world applications.

###  4.2 Continuous prompt vectors

Advancements in prompt engineering have enabled more effective adaptation of pre-trained VLMs to a wide range of downstream tasks. A promising approach in this domain is the use of continuous prompt vectors to fine-tune models such as CLIP for complex video understanding tasks. Unlike traditional handcrafted prompts, which require expert knowledge and manual effort, continuous prompt vectors [[134](https://arxiv.org/html/2310.14735v5#bib.bib134)] are learned during the training process, allowing for more flexible and efficient model adaptation. This method involves appending or prepending sequences of random vectors to the input text, which the model then interprets as part of its textual input. These vectors are optimized to effectively bridge the gap between the static image-based pre-training objectives and the dynamic requirements of video tasks, such as action recognition, action localization, and text-video retrieval. Additionally, lightweight temporal modeling using Transformers is applied to capture the temporal dependencies inherent in video data.

The efficiency of this approach lies in its minimal computational requirements; only a few parameters are trained, while the core model remains frozen. Despite this, the method has demonstrated competitive performance across various benchmarks, highlighting its potential in extending the capabilities of VLMs to handle resource-intensive video tasks with greater flexibility and accuracy. This continuous prompt-based adaptation represents a significant step forward in the evolution of prompt engineering, offering a scalable and effective solution for leveraging pre-trained models in more complex and diverse applications [[134](https://arxiv.org/html/2310.14735v5#bib.bib134)].

###  4.3 Context optimization

Context Optimization (CoOp) [[135](https://arxiv.org/html/2310.14735v5#bib.bib135)] is an innovative prompt learning approach specifically designed for VLMs. CoOp focuses on enhancing the adaptability and performance of these models by optimizing context-specific prompts. This methodology involves the creation of learnable context vectors that are embedded within the model’s architecture, enabling it to dynamically adjust to different downstream tasks.

CoOp leverages the dual-stream architecture of VLMs, such as CLIP [[16](https://arxiv.org/html/2310.14735v5#bib.bib16)] and ALIGN [[136](https://arxiv.org/html/2310.14735v5#bib.bib136)], by performing context optimization on top of these pre-trained models. CoOp introduces learnable context vectors that are fine-tuned to minimize classification loss, thus avoiding extensive manual prompt engineering. By utilizing learnable context vectors, CoOp fine-tunes the prompts to align with the specific characteristics of the complex input data. This results in improved performance and better generalization across various scenarios [[137](https://arxiv.org/html/2310.14735v5#bib.bib137)]. This method is particularly valuable in applications such as image recognition and VQA, where the context can vary significantly [[138](https://arxiv.org/html/2310.14735v5#bib.bib138)].

To illustrate the practical application of CoOp, consider a VQA task [[127](https://arxiv.org/html/2310.14735v5#bib.bib127), [128](https://arxiv.org/html/2310.14735v5#bib.bib128), [129](https://arxiv.org/html/2310.14735v5#bib.bib129), [130](https://arxiv.org/html/2310.14735v5#bib.bib130)]. In a VQA scenario, the model is presented with an image and a corresponding question, and it must generate an accurate answer based on the visual and textual information. By leveraging CoOp, the model utilizes learnable context vectors to optimize the prompts specific to the context of the input image and question. This process enhances the model’s ability to interpret the visual elements and comprehend the textual query, leading to more precise and contextually relevant answers. For instance, if the model is shown an image of a beach scene with the question “What activity are the people engaged in?”, CoOp would utilize learnable context vectors to optimize the textual prompts. These context vectors help the text encoder generate features that focus on relevant aspects of the image, such as identifying people, recognizing activities, and understanding the overall context of the scene. By aligning these optimized text features with the image features extracted by the image encoder, CoOp enables the model to generate a precise and contextually relevant answer, such as “The people are playing volleyball on the beach.”

Regarding CoOp’s effectiveness, [[135](https://arxiv.org/html/2310.14735v5#bib.bib135)] showed that models using CoOp significantly outperform traditional models in tasks such as image recognition and VQA. Additionally, [[138](https://arxiv.org/html/2310.14735v5#bib.bib138)] highlighted the benefits of ensembling context optimization, which further enhances the model’s performance by combining multiple context vectors. This approach has been shown to improve the robustness and generalization of VLMs in real-world applications [[139](https://arxiv.org/html/2310.14735v5#bib.bib139)].

###  4.4 Conditional prompt learning

Conditional Context Optimization (CoCoOp) [[140](https://arxiv.org/html/2310.14735v5#bib.bib140)] is a methodology that dynamically tailors prompts based on specific conditions or contexts. Specifically, CoCoOp employs a lightweight neural network to generate input-conditional prompt vectors for each image, ensuring that the pre-trained model parameters remain unchanged. By leveraging contextual information, CoCoOp can provide more precise and relevant guidance to the model, which is particularly useful in complex, multimodal scenarios where the interplay between different types of data must be carefully managed.

One significant advantage of CoCoOp is its ability to adapt to new and unseen data without the need for fine-tuning the pre-trained model, thanks to the context-specific prompts generated by the lightweight neural network. In other words, a VLM enhanced with conditional prompts can more accurately interpret and respond to images and questions it has not encountered during training [[140](https://arxiv.org/html/2310.14735v5#bib.bib140), [141](https://arxiv.org/html/2310.14735v5#bib.bib141)]. This capability is critical for applications such as image captioning, VQA, and scene understanding, where the context can vary widely.

Consider an image captioning task where the goal is to generate descriptive captions for images. Using CoCoOp, the model enhances its performance with dynamically generated prompts tailored for different types of scenes. Specifically, CoCoOp extends the CoOp method by training a lightweight neural network to generate input-conditional tokens for each image. As detailed in [[140](https://arxiv.org/html/2310.14735v5#bib.bib140)], this allows the model to adapt to various contexts without extensive retraining, resulting in more accurate and contextually relevant captions. For example, a prompt for an outdoor scene might include contextual cues related to nature, weather, and activities, while a prompt for an indoor scene might focus on objects, people, and interactions. For an image of a bustling market, the conditional prompt could include cues such as “Identify the types of products being sold” or “Describe the interactions between vendors and customers”. This enables the model to produce a caption such as “Vendors selling fresh fruits and vegetables in a crowded market, with customers browsing and purchasing items” [[140](https://arxiv.org/html/2310.14735v5#bib.bib140)].

This dynamic adaptation improves caption accuracy and enhances the model’s ability to generalize to novel scenes, addressing the limitations of static prompt methods such as CoOp. Besides image captioning, the improved generalization capabilities of this technique make the model more robust in tasks such as VQA, image classification, and other real-world applications [[142](https://arxiv.org/html/2310.14735v5#bib.bib142)].

###  4.5 Multimodal prompt learning

The core idea of Multimodal Prompt Learning (MaPLe) is to introduce and optimize prompts for both the vision and language components simultaneously. By embedding prompts at various stages within the transformer architecture, MaPLe ensures that the model can adaptively learn contextual information pertinent to the specific task at hand [[143](https://arxiv.org/html/2310.14735v5#bib.bib143)]. This hierarchical approach allows the model to progressively refine its understanding and integration of multimodal inputs, leading to improved performance across a range of applications.

One of the critical innovations of MaPLe is its ability to enhance task relevance. Traditional prompt engineering often focuses on either vision or language prompts in isolation, which can limit the model’s ability to fully leverage the complementary information available in multimodal data. MaPLe overcomes this limitation by jointly optimizing prompts for both modalities, thereby facilitating a more integrated and coherent representation of the input data [[143](https://arxiv.org/html/2310.14735v5#bib.bib143), [141](https://arxiv.org/html/2310.14735v5#bib.bib141)]. A detailed comparison between the MaPLe and traditional method (CoOp and CoCoOp) is shown in Figure [17](https://arxiv.org/html/2310.14735v5#S4.F17 "Figure 17 ‣ 4.5 Multimodal prompt learning ‣ 4 Methodologies for multimodal large models ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review").

![Refer to caption](x8.png) Figure 17: Comparison between the MaPLe and traditional method (CoOp and CoCoOp).

Another important mechanism of MaPLe, the hierarchical learning mechanism, allows the model to process and integrate information at multiple levels of abstraction. This is particularly beneficial for complex tasks that require a deep understanding of both visual and textual elements. By optimizing prompts at different layers within the transformer, MaPLe can better capture the intricate dependencies between vision and language inputs [[143](https://arxiv.org/html/2310.14735v5#bib.bib143), [144](https://arxiv.org/html/2310.14735v5#bib.bib144)].

[[143](https://arxiv.org/html/2310.14735v5#bib.bib143)] showed that MaPLe significantly outperforms baseline models in tasks such as image recognition and VQA. Similarly, [[141](https://arxiv.org/html/2310.14735v5#bib.bib141)] highlighted the importance of Multimodal prompt learning in enhancing the adaptability and generalization of VLMs.

To illustrate the practical application of MaPLe, consider the task of VQA [[127](https://arxiv.org/html/2310.14735v5#bib.bib127), [128](https://arxiv.org/html/2310.14735v5#bib.bib128), [129](https://arxiv.org/html/2310.14735v5#bib.bib129), [130](https://arxiv.org/html/2310.14735v5#bib.bib130)]. In a typical VQA scenario, a model is provided with an image and a related question, and it must generate a correct and contextually relevant answer. Using MaPLe, the model can be fine-tuned with multimodal prompts that simultaneously address both the visual content and the textual question. For instance, given an image of a bustling market and the question “What fruit is the vendor selling?”, MaPLe would embed prompts at various levels of the transformer’s vision and language branches. These prompts might include visual prompts that focus on identifying objects and text prompts that guide the model to look for specific answer-relevant details. By processing these prompts hierarchically, the model can effectively integrate visual cues (like recognizing apples and oranges in the image) with the textual context (understanding the question) to generate an accurate answer (e.g., “The vendor is selling apples and oranges”). This multimodal approach ensures that the model leverages both the visual and textual information in a coherent and integrated manner, resulting in improved performance on VQA tasks compared to models that do not utilize such comprehensive prompt learning strategies.

##  5 Assessing the efficacy of prompt methods

There exist several ways to evaluate the quality of the output of an LLM. Evaluation methods can generally be divided into subjective and objective categories to assess the efficacy of current prompt methods in AIGC tools

###  5.1 Subjective and objective evaluations

The task of prompt engineering can be challenging because it is difficult to determine how a prompt is more effective solely based on its raw text form [[145](https://arxiv.org/html/2310.14735v5#bib.bib145)]. Therefore, evaluating prompts requires a combination of subjective and objective methods. Subjective evaluations primarily rely on human evaluators to assess the quality of the generated content. Objective evaluations, also known as automatic evaluation methods, use algorithms to score the quality of text generated by LLMs or test on various benchmarks to quantitatively measure the efficacy of prompt methods.

Subjective evaluation and objective evaluation methods each have their advantages and disadvantages. Subjective evaluation is more in line with human intuition, but it is also more expensive and time-consuming [[146](https://arxiv.org/html/2310.14735v5#bib.bib146)]. Objective evaluation is less expensive and quicker than subjective evaluation. For instance, despite numerous pieces of research highlighting the limited correlation between BLEU and alternative metrics based on human assessments, their popularity has remained unaltered [[147](https://arxiv.org/html/2310.14735v5#bib.bib147), [148](https://arxiv.org/html/2310.14735v5#bib.bib148)]. The best way to evaluate the quality of LLM output depends on the specific application [[149](https://arxiv.org/html/2310.14735v5#bib.bib149)].

####  5.1.1 Subjective evaluations

Subjective evaluations depend on human evaluators to judge the quality of the generated content. Human evaluators can read the text generated by LLMs and score it for quality. Subjective evaluations typically include aspects such as fluency, accuracy, novelty, and relevance [[34](https://arxiv.org/html/2310.14735v5#bib.bib34)]. [[150](https://arxiv.org/html/2310.14735v5#bib.bib150)] builds a human evaluation for their “Chain of Density” (CoD) method based on “good summary” standard [[151](https://arxiv.org/html/2310.14735v5#bib.bib151)]. The four writers of the paper scored 100 summaries which include randomly shuffled CoD summaries to evaluate the performance. [[77](https://arxiv.org/html/2310.14735v5#bib.bib77)] using human judgments to compare outputs from other methods and “tree-of-thought” by asking the model to finish creative writing. They averaged the score for each output and found that the score from human judgment was consistent, which means that the results from human judges are credible. [[152](https://arxiv.org/html/2310.14735v5#bib.bib152)] invites 3 human annotators to create a set to explore the alignment between human and automatic evaluation. [[146](https://arxiv.org/html/2310.14735v5#bib.bib146)] assesses the quality with three human judges who indicated whether the generated norms and moral actions were relevant to the given moral story. Above these, subjective evaluations are increasingly used to assess content generated by models in areas that are difficult to represent with datasets and are more abstract, such as writing and summary.

####  5.1.2 Objective evaluations

Objective evaluations, also known as automatic evaluation methods, use algorithms to assess the quality of content generated by LLMs or to conduct tests on various benchmarks, quantitatively measuring the effectiveness of different prompt methods. Human-AI Language-based Interaction Evaluation (HALIE) [[153](https://arxiv.org/html/2310.14735v5#bib.bib153)], components of human-LM interactive systems and evaluation metrics, putting interaction at the center of LM evaluation. One kind of objective evaluation employs automated metrics, such as BiLingual Evaluation Understudy (BLEU) [[154](https://arxiv.org/html/2310.14735v5#bib.bib154)], which assigns a score to system-generated outputs, offering a convenient and rapid way to compare various systems and monitor their advancements. Other evaluations such as Recall-Oriented Understudy for Gisting Evaluation (ROUGE) [[155](https://arxiv.org/html/2310.14735v5#bib.bib155)], and Metric for Evaluation of Translation with Explicit ORdering (METEOR) [[156](https://arxiv.org/html/2310.14735v5#bib.bib156)], assess the similarity between the generated text and reference text. More recent evaluation methods, such as BERTScore [[157](https://arxiv.org/html/2310.14735v5#bib.bib157)], aim to assess at a higher semantic level.

However, these automated metrics often fail to capture the assessment results of human evaluators fully and therefore must be used with caution [[158](https://arxiv.org/html/2310.14735v5#bib.bib158)]. So many researchers evaluate their methods by quantitating the performance of the model under specific tasks. Some of the tasks are traditional games, such as Game of 24 and 5x5 Crosswords [[77](https://arxiv.org/html/2310.14735v5#bib.bib77)]. The other tasks, in other words, called benchmarks, are datasets that contain instructions for models to finish. Exclude the comprehensive set of benchmarks such as Beyond the Imitation Game benchmark (BIGbench) [[159](https://arxiv.org/html/2310.14735v5#bib.bib159)] and Big-Bench Hard (BBH) [[160](https://arxiv.org/html/2310.14735v5#bib.bib160)], which evaluates the logical soundness of arguments, there are four kinds of benchmarks concluded below. These benchmarks provide standardized tasks and datasets that facilitate consistent and comparable assessments of different approaches. For testing prompt engineering methods, it is not to pursue the ”best” benchmark but to choose the one that is most suitable for evaluating the model’s abilities, because not a single model can perform best in all kinds of tasks [[161](https://arxiv.org/html/2310.14735v5#bib.bib161)].

##### Math Word Problems (MWPs)

Objective evaluations about MWP test a model’s ability to understand numerical-related questions. The task is challenging because the model needs to understand relevant information from natural language text as well as perform mathematical reasoning to solve it. The complexity of MWPs can be measured along multiple axes, e.g., reasoning and linguistic complexity and world and domain knowledge. Similar to earlier benchmark MATH23K [[162](https://arxiv.org/html/2310.14735v5#bib.bib162)] and Hybrid Math Word Problems dataset (HMWP) [[163](https://arxiv.org/html/2310.14735v5#bib.bib163)], simple Variations on Arithmetic Math word Problems (SVAMP) [[76](https://arxiv.org/html/2310.14735v5#bib.bib76)] is a kind of MWP benchmark to solve elementary-level math word problems, which evaluates the performance of models by asking them to give equations and answers based on the questions in elementary school. Dolphin1878 [[164](https://arxiv.org/html/2310.14735v5#bib.bib164)] is a kind of number-word problem over 1,500 number-word problems. ARIS [[165](https://arxiv.org/html/2310.14735v5#bib.bib165)] and AllArith [[166](https://arxiv.org/html/2310.14735v5#bib.bib166)] are arithmetic word problems and MAth Word ProblemS (MAWPs) [[167](https://arxiv.org/html/2310.14735v5#bib.bib167)] present algebraic word problems to test problem-solving skills. Different from these benchmarks contain one category of field, Academia Sinica Diverse MWP Dataset (ASDiv) [[168](https://arxiv.org/html/2310.14735v5#bib.bib168)], Algebra Question Answering (AQuA) [[169](https://arxiv.org/html/2310.14735v5#bib.bib169)] and MathQA [[170](https://arxiv.org/html/2310.14735v5#bib.bib170)] including more domains than others, such as arithmetic, algebraic and domain knowledge problems. SingleEQ [[171](https://arxiv.org/html/2310.14735v5#bib.bib171)] is construed with both single-step and multi-step math problems from mixed sources. MultiArith [[172](https://arxiv.org/html/2310.14735v5#bib.bib172)] includes elementary math problems with multiple steps. MATH [[173](https://arxiv.org/html/2310.14735v5#bib.bib173)] and GSM8K [[60](https://arxiv.org/html/2310.14735v5#bib.bib60)] require models to solve complex mathematical problems, emphasizing the need for a deep understanding of mathematical concepts and reasoning. Process-supervised Reward Models (PRM) 800K [[174](https://arxiv.org/html/2310.14735v5#bib.bib174)] includes 4.5K MATH test problems, and contains about 800,000 step-level labels over 75,000 solutions.

##### Question Answering (QA) Tasks

QA tasks require models to return feedback due to the given question. Massive Multitask Language Understanding (MMLU) [[175](https://arxiv.org/html/2310.14735v5#bib.bib175)] is a QA benchmark designed to measure knowledge acquired during pretraining by evaluating models exclusively in zero-shot and few-shot settings. Many QA benchmarks are also related to knowledge-based tasks. Fact Extraction and VERification (FEVER) [[176](https://arxiv.org/html/2310.14735v5#bib.bib176)] focuses on fact verification, requiring models to act for claims generated by altering sentences extracted from Wikipedia. MIDTERMQA [[177](https://arxiv.org/html/2310.14735v5#bib.bib177)] focuses on the 2022 U.S. midterm elections since the knowledge cutoff of black-box LLMs is often 2021 or earlier. These benchmarks play a critical role in assessing the models’ abilities to comprehend, analyze, and synthesize information from diverse sources. NarrativeQA [[178](https://arxiv.org/html/2310.14735v5#bib.bib178)] built by materials such as movies and books, with nearly 63k tokens of input in each question. The Question Answering with Long Input Text, Yes (QuALITY) [[179](https://arxiv.org/html/2310.14735v5#bib.bib179)] is a multiple-choice QA dataset containing 2k–8k tokens from English source articles. CommonsenseQA [[180](https://arxiv.org/html/2310.14735v5#bib.bib180), [181](https://arxiv.org/html/2310.14735v5#bib.bib181)] focuses on commonsense question answering based on ConceptNet 5.5 [[182](https://arxiv.org/html/2310.14735v5#bib.bib182)], an open multilingual graph of general knowledge. HotPotQA [[183](https://arxiv.org/html/2310.14735v5#bib.bib183)] is collected by crowdsourcing such as Wikipedia articles and AI2 Reasoning Challenge (ARC) [[184](https://arxiv.org/html/2310.14735v5#bib.bib184)] includes 14M science sentences, 787 science questions, all non-diagram, and multiple choices. GovReport [[185](https://arxiv.org/html/2310.14735v5#bib.bib185)] dataset focuses on summarizing complex government reports, testing the models’ ability to distill and synthesize critical information. QA benchmarks challenge models’ reasoning and use of commonsense knowledge ability.

##### Language Understanding Tasks

In early efforts for language understanding and inductive tasks, Text REtrieval Conference (TREC) [[186](https://arxiv.org/html/2310.14735v5#bib.bib186)] focuses on the problem of retrieving answers rather than document lists. Stanford Sentiment Treebank (SST) [[187](https://arxiv.org/html/2310.14735v5#bib.bib187)] is constructed with fully labeled parse trees, enabling a comprehensive analysis of the compositional effects of sentiment in language and named SST-2 & SST-5 based on its number of labels. Summarization tasks, as tested by datasets like SummScreenFD [[188](https://arxiv.org/html/2310.14735v5#bib.bib188)] measure the effectiveness of the methods in catching essential information from large content. AG’s News [[189](https://arxiv.org/html/2310.14735v5#bib.bib189)] is a subset of the larger AG’s Corpus which is built by compiling titles and description fields from articles belonging to different categories in AG’s Corpus. By pairing varied task instructions with the corresponding text, SentiEval [[190](https://arxiv.org/html/2310.14735v5#bib.bib190)] decreases the sensitivities associated with prompt design during the evaluation of different LLMs. CR [[191](https://arxiv.org/html/2310.14735v5#bib.bib191)], the sentiment of sentences mined from customer reviews, and MR [[192](https://arxiv.org/html/2310.14735v5#bib.bib192)], a movie review snippet sentiment on a five-star scale, are benchmarks that instruct models to classify sentiment from contents. “Less Likely Brainstorming” [[193](https://arxiv.org/html/2310.14735v5#bib.bib193)] is a benchmark that tests by asking the model to generate outputs that humans think are relevant but less likely to happen. Subj [[192](https://arxiv.org/html/2310.14735v5#bib.bib192)] is the benchmark including the subjectivity of sentences from movie reviews and plot summaries. SAlient Long-Tail Translation Error Detection (SALTED) [[194](https://arxiv.org/html/2310.14735v5#bib.bib194)] focuses on identifying errors in translations, emphasizing linguistic proficiency and attention to detail. These evaluations highlight the models’ ability to understand and process text, making accurate predictions based on the content. Coin Flip [[23](https://arxiv.org/html/2310.14735v5#bib.bib23)] dataset assesses symbolic reasoning that asks the model to answer whether a coin still heads up after people either flip or don’t flip the coin.

##### Multimodal Tasks

Multimodal tasks are designed to evaluate a MMLMs ability to process and integrate information from multiple sources, such as text and images. RefCOCO, RefCOCO+ [[195](https://arxiv.org/html/2310.14735v5#bib.bib195)] and RefCOCOg [[196](https://arxiv.org/html/2310.14735v5#bib.bib196)] provide referring expressions for objects in images, testing models’ ability to link descriptions with visual content. These evaluations are crucial for developing models capable of cross-modal understanding and interaction, essential for applications like visual question answering and image captioning.

###  5.2 Comparing different prompt methods

Some models are used to evaluate the performance of other models [[197](https://arxiv.org/html/2310.14735v5#bib.bib197), [198](https://arxiv.org/html/2310.14735v5#bib.bib198)]. The performance scores derived from different methods serve as benchmarks for evaluating models. LLM-Eval [[199](https://arxiv.org/html/2310.14735v5#bib.bib199)] is developed to measure open-domain conversations with LLMs. This method tries to evaluate the performance of LLMs on various benchmark datasets [[200](https://arxiv.org/html/2310.14735v5#bib.bib200)] such as Dynabench [[201](https://arxiv.org/html/2310.14735v5#bib.bib201)] and demonstrate their efficiency. [[202](https://arxiv.org/html/2310.14735v5#bib.bib202), [203](https://arxiv.org/html/2310.14735v5#bib.bib203), [124](https://arxiv.org/html/2310.14735v5#bib.bib124), [146](https://arxiv.org/html/2310.14735v5#bib.bib146)] compare their methods of prompt engineering with previous prompt methods such as CoT, Zero-shot, Natural Instructions (NI) [[204](https://arxiv.org/html/2310.14735v5#bib.bib204)], APO [[205](https://arxiv.org/html/2310.14735v5#bib.bib205)] and APE [[206](https://arxiv.org/html/2310.14735v5#bib.bib206)] though benchmarks such as SVAMP [[76](https://arxiv.org/html/2310.14735v5#bib.bib76)], GSM8K [[60](https://arxiv.org/html/2310.14735v5#bib.bib60)], ASDiv [[168](https://arxiv.org/html/2310.14735v5#bib.bib168)], AQUA [[169](https://arxiv.org/html/2310.14735v5#bib.bib169)], MultiArith [[172](https://arxiv.org/html/2310.14735v5#bib.bib172)], SingleEQ [[171](https://arxiv.org/html/2310.14735v5#bib.bib171)] and BBH [[160](https://arxiv.org/html/2310.14735v5#bib.bib160)]. Specific benchmarks are used to test the improvements of new prompt methods over the original model. [[207](https://arxiv.org/html/2310.14735v5#bib.bib207)] chooses QuALITY, SummScreenFD and GovReport under original type and long content type to compare with other methods such as Recurrence [[208](https://arxiv.org/html/2310.14735v5#bib.bib208), [209](https://arxiv.org/html/2310.14735v5#bib.bib209), [210](https://arxiv.org/html/2310.14735v5#bib.bib210)] and Retrieval [[211](https://arxiv.org/html/2310.14735v5#bib.bib211), [212](https://arxiv.org/html/2310.14735v5#bib.bib212)]. [[213](https://arxiv.org/html/2310.14735v5#bib.bib213)] compared their methods with APE and MI [[190](https://arxiv.org/html/2310.14735v5#bib.bib190)] by ROUGE-1, ROUGE-2 and ROUGE-L [[155](https://arxiv.org/html/2310.14735v5#bib.bib155)]. [[214](https://arxiv.org/html/2310.14735v5#bib.bib214)] calculates the score by the approach provided from [[215](https://arxiv.org/html/2310.14735v5#bib.bib215)] and compared with ReAct. [[216](https://arxiv.org/html/2310.14735v5#bib.bib216)] received better performance than other methods under RefCOCO [[195](https://arxiv.org/html/2310.14735v5#bib.bib195)], RefCOCO+ and RefCOCOg [[196](https://arxiv.org/html/2310.14735v5#bib.bib196)].

Besides comparing different methods by score, other indicators can provide additional insights. [[217](https://arxiv.org/html/2310.14735v5#bib.bib217)] adopts prediction accuracy and proof accuracy to demonstrate the advantage of Reasoning via Planning (RAP) compared with CoT under GSM8k. [[218](https://arxiv.org/html/2310.14735v5#bib.bib218)] considered the economic cost when utilizing various prompt methods. [[219](https://arxiv.org/html/2310.14735v5#bib.bib219)] reports that their Skeleton-of-Thought (SoT) method achieves nearly twice the evaluation speed. [[119](https://arxiv.org/html/2310.14735v5#bib.bib119)] divides the evaluation into 7 domains such as “dialogue”, “Slot Filling”, “Open-domain QA” that more comprehensively compare the ability to solve tasks. [[220](https://arxiv.org/html/2310.14735v5#bib.bib220)] normalizes the score of “accuracy”, “precision” and “recall” to compare their method Chain-of-Symbol Prompting (CoS) with CoT. [[221](https://arxiv.org/html/2310.14735v5#bib.bib221)] evaluates different methods by four parts-“human”, “social”, “STEM” and “other”.

Subjective comparison is also used in prompt methods comparison. [[222](https://arxiv.org/html/2310.14735v5#bib.bib222)] introduces the human-rater as a metric of evaluation. [[223](https://arxiv.org/html/2310.14735v5#bib.bib223)] compares its method “Planning and Executable Actions for Reasoning over Long documents (Pearl)” with other methods such as CoT, Program-of-Thought (PoT) [[224](https://arxiv.org/html/2310.14735v5#bib.bib224)], Self-Asked [[225](https://arxiv.org/html/2310.14735v5#bib.bib225)], Toolformer [[226](https://arxiv.org/html/2310.14735v5#bib.bib226)] and ReAct in four domains, which are “explicit plan”, “iterative prompting”, “does not rely on external tools” and ‘Long documents”. [[152](https://arxiv.org/html/2310.14735v5#bib.bib152)] combines human and automatic evaluations to assess whether the method aligns with human reasoning. [[77](https://arxiv.org/html/2310.14735v5#bib.bib77)] compares CoT with ToT by human-rated “creative writing” task.

Other studies experiment mainly on certain models or tasks and employ disparate evaluation metrics, restricting comparability across methods [[96](https://arxiv.org/html/2310.14735v5#bib.bib96), [227](https://arxiv.org/html/2310.14735v5#bib.bib227)]. Nevertheless, recent research proposed a general evaluation framework called InstructEval [[228](https://arxiv.org/html/2310.14735v5#bib.bib228)] that enables a comprehensive assessment of prompting techniques across multiple models and tasks. InstructEval reached the following conclusions: in few-shot settings, omitting prompts or using generic task-agnostic prompts tends to outperform other methods, with prompts having little impact on performance; in zero-shot settings, expert-written task-specific prompts can significantly boost performance, with automated prompts not outperforming simple baselines; the performance of automated prompt generation methods is inconsistent, varying across different models and task types, displaying a lack of generalization.

##  6 Applications improved by prompt engineering

The output enhancements provided by prompt engineering techniques make LLMs better applicable to real-world applications. This section briefly discusses applications of prompt engineering in fields such as teaching, programming, and others.

![Refer to caption](x9.png) Figure 18: Guideline of courses generated by GPT-4

###  6.1 Assessment in teaching and learning

[[229](https://arxiv.org/html/2310.14735v5#bib.bib229)] investigates the application of machine learning methods in young student education. In such a context, prompt engineering can facilitate the creation of personalized learning environments. By offering tailored prompts, LLMs can adapt to an individual’s learning pace and style. Such an approach can allow for personalized assessments and educational content, paving the way for a more individual-centric teaching model. Recent advancements in prompt engineering suggest that AI tools can also cater to students with specific learning needs, thus fostering inclusivity in education [[230](https://arxiv.org/html/2310.14735v5#bib.bib230)]. As a simple example, it is possible for professors to provide rubrics or guidelines for a future course with the assistance of AI. As Figure [18](https://arxiv.org/html/2310.14735v5#S6.F18 "Figure 18 ‣ 6 Applications improved by prompt engineering ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") shows, when GPT-4 was required to provide a rubric about a course, with a suitable prompt, it was able to respond with a specific result that may satisfy the requirement.

The advancements in prompt engineering also bring better potential for automated grading in education. With the help of sophisticated prompts, LLMs can provide preliminary assessments, reducing the workload for educators while providing instant feedback to students [[231](https://arxiv.org/html/2310.14735v5#bib.bib231)]. Similarly, these models, when coupled with well-designed prompts, can analyze a vast amount of assessment data, thus providing valuable insights into learning patterns and informing educators about areas that require attention or improvement [[232](https://arxiv.org/html/2310.14735v5#bib.bib232), [233](https://arxiv.org/html/2310.14735v5#bib.bib233)].

###  6.2 Content creation and editing

With controllable improved input, LLMs have primarily been used in creative works, such as content creation. Pathways Language Model (PaLM) [[67](https://arxiv.org/html/2310.14735v5#bib.bib67)] and prompting approach have been used to facilitate cross-lingual short story generation. The Recursive Reprompting and Revision framework (Re3superscriptRe3\mathrm{Re}^{3}roman_Re start_POSTSUPERSCRIPT 3 end_POSTSUPERSCRIPT) [[234](https://arxiv.org/html/2310.14735v5#bib.bib234)] employs zero-shot prompting [[47](https://arxiv.org/html/2310.14735v5#bib.bib47)] with GPT-3 to craft a foundational plan including elements such as settings, characters, and outlines. Subsequently, it adopts a recursive technique, dynamically prompting GPT-3 to produce extended story continuations. For another example, Detailed Outline Control (DOC) [[235](https://arxiv.org/html/2310.14735v5#bib.bib235)] aims at preserving plot coherence across extensive texts generated with the assistance of GPT-3. Unlike Re3superscriptRe3\mathrm{Re}^{3}roman_Re start_POSTSUPERSCRIPT 3 end_POSTSUPERSCRIPT, DOC employs a detailed outliner and detailed controller for implementation. The detailed outliner initially dissects the overarching outline into subsections through a breadth-first method, where candidate generations for these subsections are generated, filtered, and subsequently ranked. This process is similar to the method of chain-of-though (subsection [3.1](https://arxiv.org/html/2310.14735v5#S3.SS1 "3.1 Chain-of-thought ‣ 3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review")). Throughout this generation process, an OPT-based Future Discriminators for Generation (FUDGE) [[236](https://arxiv.org/html/2310.14735v5#bib.bib236)] detailed controller plays a crucial role in maintaining relevance.

###  6.3 Computer programming

Prompt engineering can help LLMs perform better at outputting programming codes. By using a self-debugging prompting approach [[57](https://arxiv.org/html/2310.14735v5#bib.bib57)], which contains simple feedback, unit-test, and code explanation prompts module, the text-to-SQL [[237](https://arxiv.org/html/2310.14735v5#bib.bib237)] model is able to provide a solution it can state as correct unless the maximum number of attempts has been reached. Another example, Multi-Turn Programming Benchmark (MTPB) [[238](https://arxiv.org/html/2310.14735v5#bib.bib238)], was constructed to implement a program by breaking it into multi-step natural language prompts.

Another approach is provided in [[239](https://arxiv.org/html/2310.14735v5#bib.bib239)], which introduced the Repo-Level Prompt Generator (RLPG) to dynamically retrieve relevant repository context and construct a prompt for a given task, focusing on code auto-completion tasks. The most suitable prompt is selected by a prompt proposal classifier and combined with the default context to generate the final output.

###  6.4 Reasoning tasks

AIGC tools have shown promising performance in reasoning tasks. Previous research has found that few-shot prompting can enhance the performance in generating accurate reasoning steps for word-based math problems in the GSM8K dataset [[60](https://arxiv.org/html/2310.14735v5#bib.bib60), [67](https://arxiv.org/html/2310.14735v5#bib.bib67), [55](https://arxiv.org/html/2310.14735v5#bib.bib55), [24](https://arxiv.org/html/2310.14735v5#bib.bib24)]. The strategy of including the reasoning traces in few-shot prompts [[43](https://arxiv.org/html/2310.14735v5#bib.bib43)], self-talk [[240](https://arxiv.org/html/2310.14735v5#bib.bib240)] and chain-of-thought [[23](https://arxiv.org/html/2310.14735v5#bib.bib23)], was shown to encourage the model to generate verbalized reasoning steps. [[241](https://arxiv.org/html/2310.14735v5#bib.bib241)] conducted experiments by involving prompting strategies, various fine-tuning techniques, and re-ranking methods to assess their impact on enhancing the performance of a base LLM. They found that a customized prompt significantly improved the model’s ability with fine-tuning, and demonstrated a significant advantage by generating substantially fewer errors in reasoning. In another research, [[47](https://arxiv.org/html/2310.14735v5#bib.bib47)] observed that solely using zero-shot CoT prompting leads to a significant enhancement in the performance of GPT-3 and PaLM when compared to the conventional zero-shot and few-shot prompting methods. This improvement is particularly noticeable when evaluating these models on the MultiArith [[242](https://arxiv.org/html/2310.14735v5#bib.bib242)] and GSM8K [[60](https://arxiv.org/html/2310.14735v5#bib.bib60)] datasets. [[243](https://arxiv.org/html/2310.14735v5#bib.bib243)] also introduced a novel prompting approach called Diverse Verifier on Reasoning Step (DIVERSE). This approach involves using a diverse set of prompts for each question and incorporates a trained verifier with an awareness of reasoning steps. The primary aim of DIVERSE is to enhance the performance of GPT-3 on various reasoning benchmarks, including GSM8K and others. All these works show that in the application of reasoning tasks, properly customized prompts can obtain better results from the model.

###  6.5 Dataset generation

LLMs possess the capability of in-context learning, enabling them to be effectively prompted to generate synthetic datasets for training smaller, domain-specific models. [[244](https://arxiv.org/html/2310.14735v5#bib.bib244)] put forth three distinct prompting approaches for training data generation using GPT-3: unlabeled data annotation, training data generation, and assisted training data generation. Besides, [[245](https://arxiv.org/html/2310.14735v5#bib.bib245)] is designed for the generation of supplementary synthetic data for classification tasks. GPT-3 is utilized in conjunction with a prompt that includes real examples from an existing dataset, along with a task specification. The goal is to jointly create synthetic examples and pseudo-labels using this combination of inputs.

##  7 LLMs security

Prompt engineering is the process of designing and refining the inputs (prompts) given to LLMs to elicit desired and accurate responses. This technique is crucial not only for optimizing model performance but also for enhancing security. By carefully crafting prompts, researchers and developers can identify and help to mitigate vulnerabilities in LLMs. Effective prompt engineering can expose weaknesses that might be exploited through adversarial attacks, data poisoning, or other malicious activities[[246](https://arxiv.org/html/2310.14735v5#bib.bib246)]. Conversely, poorly designed prompts can inadvertently reveal or introduce security vulnerabilities in the model [[246](https://arxiv.org/html/2310.14735v5#bib.bib246)], which could then be exploited by malicious actors, leading to issues such as the disclosure of sensitive information or susceptibility to adversarial attacks.

Thus, prompt engineering serves as both a tool for improving LLMs functionality and a critical component of their security framework. The proactive, open, and in-depth efforts of researchers in identifying and mitigating vulnerabilities through prompt engineering are essential for maintaining the integrity and safety of LLMs in diverse applications [[246](https://arxiv.org/html/2310.14735v5#bib.bib246)].

This is particularly true in critical sectors such as healthcare, finance, and cybersecurity, where prompt attacks against LLMs could lead to significant breaches of sensitive information or disrupt essential services [[247](https://arxiv.org/html/2310.14735v5#bib.bib247)]. For example, adversarial attacks can manipulate model outputs to spread harmful or misleading information [[3](https://arxiv.org/html/2310.14735v5#bib.bib3)], while data poisoning during training can corrupt the model’s learning process, leading to unreliable outputs. In healthcare, compromised models could lead to incorrect diagnoses and treatment plans, endangering patient lives. Similarly, in finance, compromised models could result in significant financial losses and undermine trust in automated financial services [[248](https://arxiv.org/html/2310.14735v5#bib.bib248)].

Consequently, there is a critical need for continuous and in-depth research in prompt engineering security to fully realize its benefits and address emerging challenges. A deeper understanding of attack methods and their mechanisms in relation to prompt engineering is essential for both large model developers and users to better defend against these threats. In this section, we will explore some mainstream attack methods related to prompt engineering and also discuss how to defend against them.

###  7.1 Adversarial attacks

Adversarial attacks involve the deliberate manipulation of input data to deceive a machine learning model into making incorrect predictions. In the context of LLMs, these attacks can take the form of subtly altered prompts or inputs that cause the model to produce unintended or harmful outputs. This manipulation exploits the sensitivity of LLMs to small perturbations in input data, revealing significant vulnerabilities [[249](https://arxiv.org/html/2310.14735v5#bib.bib249), [250](https://arxiv.org/html/2310.14735v5#bib.bib250)]. For instance, slight alterations in a text could mislead the model’s natural language understanding, leading to incorrect or biased responses [[251](https://arxiv.org/html/2310.14735v5#bib.bib251), [3](https://arxiv.org/html/2310.14735v5#bib.bib3)]. The potential for adversarial attacks is particularly concerning in applications such as automated customer service or legal document analysis, where the integrity and accuracy of responses are critical [[252](https://arxiv.org/html/2310.14735v5#bib.bib252)]. One example of adversarial attacks in image recognition is illustrated in Figure [19](https://arxiv.org/html/2310.14735v5#S7.F19 "Figure 19 ‣ 7.1 Adversarial attacks ‣ 7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review") [[253](https://arxiv.org/html/2310.14735v5#bib.bib253)].

![Refer to caption](x10.png) Figure 19: An example of adversarial attack mislead the model.

Recent research has highlighted various techniques and impacts of adversarial attacks on LLMs. Adversarial demonstration attacks, for instance, can mislead models into making incorrect predictions with only subtle changes in the input data, effectively manipulating erroneous outputs across various scenarios [[254](https://arxiv.org/html/2310.14735v5#bib.bib254)]. These incorrect predictions were verified to be due to the input data changes, and not due to the inherent randomness of the models.

Optimization techniques can enhance the effectiveness of adversarial attacks to exposes various weaknesses in LLMs, making it more challenging to defend against such threats [[255](https://arxiv.org/html/2310.14735v5#bib.bib255), [256](https://arxiv.org/html/2310.14735v5#bib.bib256)]. For instance, in legal document analysis, adversarial inputs can lead to incorrect legal interpretations, potentially affecting case outcomes. In healthcare, such attacks could mislead models into providing incorrect medical advice, jeopardizing patient safety [[257](https://arxiv.org/html/2310.14735v5#bib.bib257)]. These examples highlight the need for effective defenses against adversarial attacks to ensure the safe and reliable deployment of LLMs for such critical applications.

####  7.1.1 Data poisoning

Data poisoning involves the injection of malicious data into the training set, compromising the integrity of the model. This type of attack can significantly distort the learning process, leading to erroneous outputs once the model is deployed. In LLMs, data poisoning can be especially insidious as it may go undetected during the training phase. For instance, an attacker might insert misleading or harmful data into the large corpus used to train an LLM, causing the model to learn and reproduce these inaccuracies when prompted.

This mechanism bears some similarity with how backdoor attacks [[258](https://arxiv.org/html/2310.14735v5#bib.bib258)] can be introduced in models; both involve tampering with the training data to embed malicious patterns that influence model behavior. While backdoors typically rely on specific triggers to activate unwanted behaviors, data poisoning broadly affects the model’s overall performance and decision-making process. Although data poisoning is primarily associated with the training phase of model development, its implications extend to prompt engineering. Effective prompt engineering can help identify and mitigate the risks posed by poisoned data. For example, by carefully designing and testing prompts, practitioners can detect anomalies or unexpected model behaviors that may indicate underlying data poisoning. Moreover, prompt engineering can include rigorous data validation steps to ensure the training corpus is free from malicious alterations.

The implications of data poisoning are far-reaching, affecting sectors that rely on accurate data analysis and generation, such as healthcare, finance, and legal services [[259](https://arxiv.org/html/2310.14735v5#bib.bib259)]. Thus, integrating robust prompt engineering practices is crucial for preventing the inadvertent inclusion of poisoned data and safeguarding the reliability of LLMs.

####  7.1.2 Backdoor attacks

Backdoor threats involve embedding hidden vulnerabilities within a model that can be activated by specific prompts [[260](https://arxiv.org/html/2310.14735v5#bib.bib260)]. These backdoors, introduced during training through manipulated data, remain dormant until a trigger prompt is presented. In LLMs, a backdoor might be a specific phrase or pattern that, when encountered, triggers the model to generate a predefined, potentially harmful output, posing significant security risks due to the difficulty in detection [[258](https://arxiv.org/html/2310.14735v5#bib.bib258), [261](https://arxiv.org/html/2310.14735v5#bib.bib261), [262](https://arxiv.org/html/2310.14735v5#bib.bib262), [263](https://arxiv.org/html/2310.14735v5#bib.bib263), [264](https://arxiv.org/html/2310.14735v5#bib.bib264), [265](https://arxiv.org/html/2310.14735v5#bib.bib265), [266](https://arxiv.org/html/2310.14735v5#bib.bib266)]. A visual illustration is shown in Figure [20](https://arxiv.org/html/2310.14735v5#S7.F20 "Figure 20 ‣ 7.1.2 Backdoor attacks ‣ 7.1 Adversarial attacks ‣ 7 LLMs security ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review").

![Refer to caption](x11.png) Figure 20: An illustration of three scenarios in the backdoor attack. (1) a clean model receiving clean inputs and producing normal outputs; (2) a contaminated model receiving clean inputs but still producing normal outputs; and (3) the same model producing harmful or incorrect outputs when an implanted backdoor trigger is present.

Prompt engineering has a role to play in uncovering and mitigating backdoor threats [[267](https://arxiv.org/html/2310.14735v5#bib.bib267)]. By carefully designing and testing prompts, LLMs can be assessed for their susceptibility to adversarial attacks, data poisoning, and backdoor activations, thereby ensuring their secure deployment.

Examples of backdoor attacks include Refool [[268](https://arxiv.org/html/2310.14735v5#bib.bib268)], which leverages the natural phenomenon of reflection to stealthily implant backdoors in deep neural networks, achieving high success rates across various datasets and models while resisting state-of-the-art defenses. Similarly, [[269](https://arxiv.org/html/2310.14735v5#bib.bib269)] presents ProAttack, a clean-label backdoor method using the prompt itself as the trigger. ProAttack achieves leading attack success rates in textual backdoor attacks on LLMs, revealing critical security vulnerabilities in prompt-based learning. Building on these advancements, Imperio [[270](https://arxiv.org/html/2310.14735v5#bib.bib270)] further evolves backdoor attack strategies through language-guided instructions. By embedding backdoors during training that are later activated by natural language prompts, Imperio’s approach complicates detection, as the backdoors remain concealed and can unpredictably control model behavior, even in novel scenarios.

Backdoor threats not only pose immediate security risks but also can erode trust in AI systems, emphasizing the need for transparency in AI model development. Implementing rigorous testing frameworks to detect and eliminate hidden vulnerabilities is essential before deploying models in real-world scenarios.

####  7.1.3 Prompt injection and prompt leaking

Poorly designed prompts can make LLMs susceptible to various types of attacks, including prompt injection and prompt leaking. Prompt injection attacks involve inserting malicious inputs into prompts to manipulate the model’s output, which can result in the generation of harmful or misleading information. For example, a malicious actor could craft a prompt that subtly alters the model’s response in a way that promotes false information or biases [[252](https://arxiv.org/html/2310.14735v5#bib.bib252)]. Prompt leaking, on the other hand, occurs when sensitive or proprietary information embedded in prompts is exposed, jeopardizing the security and privacy of applications that rely on LLMs [[271](https://arxiv.org/html/2310.14735v5#bib.bib271)].

###  7.2 Prompt hacking

Prompt hacking refers to a class of attacks that involve manipulating the input prompts provided to LLMs, with the goal of provoking unintended behaviors—ranging from benign errors to severe consequences such as misinformation dissemination or data breaches. Prompt hacking exploits the fundamental way LLMs process and generate responses. Unlike traditional hacking, which exploits software vulnerabilities, prompt hacking relies on the strategic crafting of malicious inputs to deceive the LLM into performing actions that deviate from its intended function [[272](https://arxiv.org/html/2310.14735v5#bib.bib272), [273](https://arxiv.org/html/2310.14735v5#bib.bib273), [274](https://arxiv.org/html/2310.14735v5#bib.bib274), [275](https://arxiv.org/html/2310.14735v5#bib.bib275), [276](https://arxiv.org/html/2310.14735v5#bib.bib276), [277](https://arxiv.org/html/2310.14735v5#bib.bib277)]. This vulnerability is particularly concerning because it can be executed without the need for sophisticated technical skills. As LLMs become more integrated into various applications, the risk posed by prompt hacking increases, necessitating robust security measures to prevent such attacks [[278](https://arxiv.org/html/2310.14735v5#bib.bib278)].

[[279](https://arxiv.org/html/2310.14735v5#bib.bib279)] presents a large-scale study on the vulnerabilities of LLMs to prompt injection attacks by organizing a global prompt hacking competition, resulting in the creation of an extensive dataset and a comprehensive taxonomy of adversarial prompt types. [[273](https://arxiv.org/html/2310.14735v5#bib.bib273)] provides a comprehensive survey of security vulnerabilities in LLMs, focusing on prompt hacking and adversarial attacks, and discusses various defense mechanisms to enhance the resilience of these models against such threats.

Recognizing and addressing prompt hacking vulnerabilities is essential for developing robust LLM-based applications. By understanding prompt hacking techniques, developers can implement critical security measures such as stricter prompt validation, anomaly detection, and response filtering, which can collectively enhance the robustness of LLMs and mitigate the risks posed by malicious prompts [[279](https://arxiv.org/html/2310.14735v5#bib.bib279)]. Moreover, training programs for both developers and users as well as clear guidelines can significantly reduce the likelihood of successful prompt injection attacks, thus maintaining the integrity of LLM systems [[280](https://arxiv.org/html/2310.14735v5#bib.bib280)].

In response to these challenges, recent initiatives such as the OWASP LLM prompt hacking project [[281](https://arxiv.org/html/2310.14735v5#bib.bib281)] have emerged, offering not only valuable educational resources but also practical exercises to equip developers and security professionals with the tools to identify and prevent prompt hacking attacks, thereby reinforcing the security measures necessary to unleash the full potential of prompt engineering in LLMs. Practical security measures against prompt hacking include the use of advanced monitoring tools that detect suspicious prompt patterns and the integration of machine learning models trained to identify and block malicious prompts. Once again, these measures are crucial for ensuring the safe deployment of LLMs in sensitive applications such as healthcare, finance, and customer service [[282](https://arxiv.org/html/2310.14735v5#bib.bib282)].

###  7.3 Model stealing

Model stealing attacks are an adversarial misuse of prompt engineering techniques aimed at replicating the functionality or extracting proprietary knowledge from LLMs. By crafting strategically designed prompts, attackers can systematically interact with the target model, gradually reconstructing its internal mechanics or sensitive data. This process, known as “query-based extraction”, allows the attacker to build a surrogate model that mimics the target model’s responses. This gradual model reconstruction approach relies on the ability to generate diverse and informative prompts that cover a wide range of inputs the model might encounter [[283](https://arxiv.org/html/2310.14735v5#bib.bib283)], and can be particularly effective when the target model is a black-box system, where the attacker has no access to the internal architecture but can observe the outputs generated in response to the inputs [[284](https://arxiv.org/html/2310.14735v5#bib.bib284)].

This stealing process highlights the vulnerabilities inherent in LLMs when exposed to malicious prompt manipulations, potentially resulting in intellectual property theft, erosion of competitive advantages, and the unethical deployment of cloned models in unauthorized contexts [[285](https://arxiv.org/html/2310.14735v5#bib.bib285), [286](https://arxiv.org/html/2310.14735v5#bib.bib286), [287](https://arxiv.org/html/2310.14735v5#bib.bib287)].

One notable example of a model stealing attack is the extraction of the projection matrix from OpenAI’s language models. Researchers demonstrated how, through a series of carefully crafted prompts, they could extract significant portions of the model’s architecture and parameters, effectively creating a replica of the original model [[288](https://arxiv.org/html/2310.14735v5#bib.bib288)]. Another incident involved adversaries using prompt engineering techniques to replicate commercial LLMs used in customer service, resulting in substantial intellectual property theft and financial losses for the companies involved [[289](https://arxiv.org/html/2310.14735v5#bib.bib289)].

Numerous studies have explored model stealing of LLMs. For instance, [[290](https://arxiv.org/html/2310.14735v5#bib.bib290)] proposes a novel prompt stealing attack against LLMs by introducing a two-stage approach involving parameter extraction and prompt reconstruction, effectively demonstrating the vulnerability of LLMs to reverse engineering of prompts based on their generated responses. Furthermore, [[291](https://arxiv.org/html/2310.14735v5#bib.bib291)] introduces PRSA, a novel framework for prompt stealing attacks against LLMs, which effectively infers the intent of target prompts and generates functionally equivalent surrogate prompts, highlighting the significant risks of prompt leakage in both non-interactive and interactive prompt services.

The effectiveness of these attacks underscores the need for robust defenses. Proposed countermeasures include limiting the number of queries a single user can make, implementing anomaly detection to identify suspicious querying patterns, and using defensive perturbations to mislead potential attackers [[292](https://arxiv.org/html/2310.14735v5#bib.bib292)].

###  7.4 Enhancing LLMs security

Adversarial example generation is a fundamental technique in AI security, designed to test and enhance the robustness of machine learning models. By creating inputs that intentionally mislead models into making incorrect predictions, researchers can identify vulnerabilities and develop strategies to mitigate them. This process is essential for ensuring that models can withstand malicious attacks and function reliably in real-world scenarios [[252](https://arxiv.org/html/2310.14735v5#bib.bib252)]. While direct research on prompt-based adversarial example generation is limited, prompt engineering remains a critical tool in testing model robustness. By designing prompts that subtly alter input data, researchers can simulate adversarial conditions and observe how models respond. For example, ambiguous or misleading prompts can reveal how susceptible a model is to producing biased or incorrect outputs, thereby identifying potential weaknesses [[293](https://arxiv.org/html/2310.14735v5#bib.bib293)].

Adversarial training, which involves training models on adversarial examples, has proven effective in enhancing model robustness [[294](https://arxiv.org/html/2310.14735v5#bib.bib294)]. [[285](https://arxiv.org/html/2310.14735v5#bib.bib285)] have shown that models exposed to a variety of adversarial inputs during training are better equipped to handle unexpected or malicious data. This method improves the resilience of models against attacks and enhances their overall reliability.

To maximize the effectiveness of adversarial training, integrating robust prompt design is essential. This involves creating prompts that not only test the model’s limits but also enhance its ability to learn from adversarial conditions. Techniques such as mask filling, where portions of text are strategically manipulated, can be used to generate adversarial examples that expose and address vulnerabilities in the model [[255](https://arxiv.org/html/2310.14735v5#bib.bib255)].

To conclude this section on security, while prompt engineering can greatly enhance the capabilities of LLMs, it can also introduce significant risks if not managed properly. The current race to release new and improved LLM functionalities cannot disregard the critical need for secure and robust design practices to combat adversarial prompts. As LLMs become further embedded in critical applications, advancing secure prompt engineering practices is essential to safeguard against misuse, minimize security vulnerabilities, and ensure safe deployment [[295](https://arxiv.org/html/2310.14735v5#bib.bib295), [3](https://arxiv.org/html/2310.14735v5#bib.bib3)].

##  8 Prospective methodologies

Beyond the advanced methodologies discussed in Section [3](https://arxiv.org/html/2310.14735v5#S3 "3 Advanced methodologies ‣ Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review"), several key developments on the horizon promise to substantially advance prompt engineering capabilities. This brief section discusses some noteworthy trajectories, which could shape the future of prompt engineering.

###  8.1 Better understanding of structures

One significant trajectory about the future of prompt engineering that emerges is the importance of better understanding the underlying structures of AI models. This understanding is crucial to effectively guide these models through prompts and to generate outputs that are more closely aligned with user intent.

At the heart of most AI models, including GPT-4, are complex mechanisms designed to understand and generate human language. The interplay of these mechanisms forms the “structure” of these models. Understanding this structure involves unraveling the many layers of neural networks, the various attention mechanisms at work, and the role of individual nodes and weights in the decision-making process of these models [[296](https://arxiv.org/html/2310.14735v5#bib.bib296)]. Deepening our understanding of these structures could lead to substantial improvements in prompt engineering. The misunderstanding of the model may cause a lack of reproducibility [[297](https://arxiv.org/html/2310.14735v5#bib.bib297)]. By understanding how specific components of the model’s structure influence its outputs, one could design prompts that more effectively exploit these components.

Furthermore, a comprehensive grasp of these structures could shed light on the shortcomings of certain prompts and guide their enhancement. Frequently, the underlying causes for a prompt’s inability to yield the anticipated output are intricately linked to the model’s architecture. For example, [[22](https://arxiv.org/html/2310.14735v5#bib.bib22)] found evidence of limitations in previous prompt models and questioned how much these methods truly understood the model.

The exploration of AI model architectures remains a vibrant research domain, with numerous endeavors aimed at comprehending these sophisticated frameworks. A notable instance is DeepMind’s “Causal Transformer” model [[298](https://arxiv.org/html/2310.14735v5#bib.bib298)], designed to explicitly delineate causal relationships within data. This represents a stride towards a more profound understanding of AI model architectures, with the potential to help people design more efficient prompts.

Along the same lines, a more comprehensive grasp of AI model architectures would also yield advancements in explainable AI. Beyond better prompt engineering, this would also foster greater trust in AI systems and promote their integration across diverse industries [[299](https://arxiv.org/html/2310.14735v5#bib.bib299)]. For example, while AI is transforming the financial sector, encompassing areas such as customer service, fraud detection, risk management, credit assessments, and high-frequency trading, several challenges, particularly those related to transparency, are emerging alongside these advancements [[300](https://arxiv.org/html/2310.14735v5#bib.bib300), [301](https://arxiv.org/html/2310.14735v5#bib.bib301)]. Another example is medicine, where AI’s transformative potential faces similar challenges [[302](https://arxiv.org/html/2310.14735v5#bib.bib302), [303](https://arxiv.org/html/2310.14735v5#bib.bib303)].

###  8.2 Agent for AIGC tools

The concept of AI agents has emerged as a potential trajectory in AI research [[304](https://arxiv.org/html/2310.14735v5#bib.bib304)]. In this brief subsection, we explore the relationship between agents and prompt engineering and project how agents might influence the future trajectory of AI-generated content (AIGC) tools. By definition, an AI agent comprises large models, memory, active planning, and tool use. AI agents are capable of remembering and understanding a vast array of information, actively planning and strategizing, and effectively using various tools to generate optimal solutions within complex problem spaces [[305](https://arxiv.org/html/2310.14735v5#bib.bib305)].

The evolution of AI agents can be delineated into five distinct phases: models, prompt templates, chains, agents, and multi-agents. Each phase carries its specific implications for prompt engineering. Foundational models, exemplified by architectures such as GPT-4, underpin the realm of prompt engineering.

In particular, prompt templates offer an effective way of applying prompt engineering in practice [[23](https://arxiv.org/html/2310.14735v5#bib.bib23)]. By using these templates, one can create standardized prompts to guide large models, making the generated output more aligned with the desired outcome. The usage of prompt templates is a crucial step towards enabling AI agents to better understand and execute user instructions.

AI agents amalgamate these methodologies and tools into an adaptive framework. Possessing the capability to autonomously modulate their behaviors and strategies, they strive to optimize both efficiency and precision in task execution. A salient challenge for prompt engineering emerges: devising and instituting prompts that adeptly steer AI agents toward self-regulation [[22](https://arxiv.org/html/2310.14735v5#bib.bib22)].

##  9 Conclusion

In conclusion, prompt engineering has established itself as an essential technique for optimizing the performance of LLMs. By employing foundational methods such as clear instructions and role-prompting, alongside advanced methodologies such as chain-of-thought and self-consistency, the capabilities of LLMs can be significantly enhanced. For VLMs, innovative strategies such as CoOp and MaPLe ensure effective integration and optimization of visual and textual data. The efficacy of these methods can be rigorously assessed through both subjective and objective evaluations, confirming their impact across diverse applications, including education, content creation, and programming. Additionally, prompt engineering has a crucial role to play in fortifying LLM security, identifying vulnerabilities, and mitigating risks through adversarial training. Looking ahead, future advancements could focus on a deeper understanding of model structures and the development of AI agents, further elevating the sophistication and capability of AI systems. This comprehensive review underscores the transformative potential of prompt engineering in advancing AI capabilities, providing a structured framework for future research and applications.

##  10 Acknowledgement

The authors would like to acknowledgde the support from the Interdisciplinary Intelligence Super Computer Center of Beijing Normal University at Zhuhai. This work was funded by the Natural Science Foundation of China (12271047); Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science, BNU-HKBU United International College (2022B1212010006); UIC research grant (R0400001-22; UICR0400008-21; R72021114; UICR0400036-21CTL; UICR04202405-21, UICR0700041-22); Guangdong College Enhancement and Innovation Program (2021ZDZX1046).

## References

  * \bibcommenthead
  * Vaswani et al. [2017] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al.  Attention is all you need.  In: Proceedings of the 31st International Conference on Neural Information Processing Systems. NIPS’17; 2017. p. 6000–6010. 
  * Bender et al. [2021] Bender EM, Gebru T, McMillan-Major A, Shmitchell S.  On the dangers of stochastic parrots: can language models be too big?  In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency; 2021. p. 610–623. 
  * Brown et al. [2020] Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, et al.  Language models are few-shot learners.  In: Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS’20; 2020. p. 1877–1901. 
  * OpenAI et al. [2024] OpenAI, Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, et al.  GPT-4 technical report; 2024.  ArXiv:2303.08774. 
  * Team et al. [2024] Team G, Anil R, Borgeaud S, Alayrac JB, Yu J, Soricut R, et al.  Gemini: a family of highly capable multimodal models; 2024.  ArXiv:2312.11805. 
  * Google [2024] Google.: Google Gemini: next-generation model.  Available from: <https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/#sundar-note>. 
  * Hulbert [2023] Hulbert D.: Tree of knowledge: ToK aka Tree of Knowledge dataset for large language models LLM.  Accessed: 2023-8-15.  <https://github.com/dave1010/tree-of-thought-prompting>. 
  * Anthropic [2024] Anthropic.  The Claude 3 model family: Opus, Sonnet, Haiku.  Anthropic; 2024.  <https://paperswithcode.com/paper/the-claude-3-model-family-opus-sonnet-haiku>. 
  * Anthropic [2024] Anthropic.: Claude 3 model.  Available from: <https://www.anthropic.com/news/claude-3-family>. 
  * Touvron et al. [2023] Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, et al.  Llama 2: open foundation and fine-tuned chat models; 2023.  ArXiv:2307.09288. 
  * Dubey et al. [2024] Dubey A, Jauhri A, Pandey A, Kadian A, Al-Dahle A, Letman A, et al.  The Llama 3 herd of models; 2024.  ArXiv2407.21783. 
  * Sarkhel et al. [2023] Sarkhel R, Huang B, Lockard C, Shiralkar P.  Self-training for label-efficient information extraction from semi-structured web-pages.  Proceedings of the VLDB Endowment. 2023;16(11):3098–3110. 
  * Ramesh et al. [2021] Ramesh A, Pavlov M, Goh G, Gray S, Voss C, Radford A, et al.  Zero-shot text-to-image generation; 2021.  ArXiv:2102.12092. 
  * Marcus et al. [2022] Marcus G, Davis E, Aaronson S.  A very preliminary analysis of DALL-E 2; 2022.  ArXiv:2204.13807. 
  * OpenAI [2021] OpenAI.: DALL·E: creating images from text.  <https://openai.com/index/dall-e/>. 
  * Radford et al. [2021] Radford A, Kim JW, Hallacy C, Ramesh A, Goh G, Agarwal S, et al.  Learning transferable visual models from natural language supervision.  In: International Conference on Machine Learning; 2021. p. 8748–8763. 
  * Li et al. [2022] Li Y, Liang F, Zhao L, Cui Y, Ouyang W, Shao J, et al.  Supervision exists everywhere: a data efficient contrastive language-image pre-training paradigm.  In: Tenth International Conference on Learning Representations; 2022. . 
  * OpenAI [2024] OpenAI.: Hello GPT-4o.  Accessed: 2024-08-04.  <https://openai.com/index/hello-gpt-4o/>. 
  * Moore [2024] Moore O.: Announcing GPT-4o in the API!  Accessed: 2024-05-22.  Available from: <https://community.openai.com/t/announcing-gpt-4o-in-the-api/744700>. 
  * Kaddour et al. [2023] Kaddour J, Harris J, Mozes M, Bradley H, Raileanu R, McHardy R.  Challenges and applications of large language models; 2023.  ArXiv:2307.10169. 
  * Lu et al. [2022] Lu Y, Bartolo M, Moore A, Riedel S, Stenetorp P.  Fantastically ordered prompts and where to find them: overcoming few-shot prompt order sensitivity.  In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics; 2022. p. 8086–8098. 
  * Webson and Pavlick [2022] Webson A, Pavlick E.  Do prompt-based models really understand the meaning of their prompts?  In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; 2022. p. 2300–2344. 
  * Wei et al. [2022] Wei J, Wang X, Schuurmans D, Bosma M, Ichter B, Xia F, et al.  Chain-of-thought prompting elicits reasoning in large language models.  In: Advances in Neural Information Processing Systems. vol. 35; 2022. p. 24824–24837. 
  * Wang et al. [2023] Wang X, Wei J, Schuurmans D, Le QV, Chi EH, Narang S, et al.  Self-consistency improves chain of thought reasoning in language models.  In: Eleventh International Conference on Learning Representations; 2023. . 
  * Shanahan et al. [2023] Shanahan M, McDonell K, Reynolds L.  Role-play with large language models; 2023.  ArXiv:2305.16367. 
  * Maynez et al. [2020] Maynez J, Narayan S, Bohnet B, McDonald R.  On faithfulness and factuality in abstractive summarization.  In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics; 2020. p. 1906–1919. 
  * Bubeck et al. [2023] Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, et al.  Sparks of artificial general intelligence: early experiments with GPT-4; 2023.  ArXiv:2303.12712. 
  * Yong et al. [2022] Yong G, Jeon K, Gil D, Lee G.  Prompt engineering for zero‐shot and few‐shot defect detection and classification using a visual‐language pretrained model.  Computer-Aided Civil and Infrastructure Engineering. 2022;38(11):1536–1554. 
  * Wang et al. [2023] Wang J, Liu Z, Zhao L, Wu Z, Ma C, Yu S, et al.  Review of large vision models and visual prompt engineering.  Meta-Radiology. 2023;1(3):100047. 
  * Christiano et al. [2017] Christiano PF, Leike J, Brown T, Martic M, Legg S, Amodei D.  Deep reinforcement learning from human preferences.  Advances in neural information processing systems. 2017;30. 
  * Stiennon et al. [2020] Stiennon N, Ouyang L, Wu J, Ziegler D, Lowe R, Voss C, et al.  Learning to summarize with human feedback.  Advances in Neural Information Processing Systems. 2020;33:3008–3021. 
  * Radford et al. [2019] Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I, et al.: Language models are unsupervised multitask learners.  Assessed: 2019-02-07.  <https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf>. 
  * Radford et al. [2018] Radford A, Narasimhan K, Salimans T, Sutskever I, et al.  Improving language understanding by generative pre-training; 2018.  Https://openai.com/research/language-unsupervised. 
  * Holtzman et al. [2020] Holtzman A, Buys J, Du L, Forbes M, Choi Y.  The curious case of neural text degeneration.  In: Ninth International Conference on Learning Representations; 2020. . 
  * Welleck et al. [2019] Welleck S, Kulikov I, Roller S, Dinan E, Cho K, Weston J.  Neural text generation with unlikelihood training; 2019.  ArXiv:1908.04319. 
  * Xu et al. [2023] Xu X, Tao C, Shen T, Xu C, Xu H, Long G, et al.  Re-reading improves reasoning in language models; 2023.  ArXiv:2309.06275. 
  * YanSong and JingLi Tencent [2018] YanSong S, JingLi Tencent A.  Joint learning embeddings for Chinese words and their components via ladder structured networks.  In: Proceedings of the Twenty-Seventh International Joint Conference on Artifificial Intelligence (IJCAI-18); 2018. p. 4375–4381. 
  * Luo et al. [2019] Luo L, Ao X, Song Y, Li J, Yang X, He Q, et al.  Unsupervised neural aspect extraction with sememes.  In: IJCAI; 2019. p. 5123–5129. 
  * Yang et al. [2019] Yang M, Qu Q, Tu W, Shen Y, Zhao Z, Chen X.  Exploring human-like reading strategy for abstractive text summarization.  In: Proceedings of the AAAI conference on artificial intelligence. vol. 33; 2019. p. 7362–7369. 
  * Zhang et al. [2023] Zhang Z, Gao J, Dhaliwal RS, Jia-Jun Li T.  VISAR: a human-AI argumentative writing assistant with visual programming and rapid draft prototyping; 2023.  ArXiv:2304.07810. 
  * Van Buren [2023] Van Buren D.  Guided scenarios with simulated expert personae: a remarkable strategy to perform cognitive work; 2023.  ArXiv:2306.03104. 
  * OpenAI [2023] OpenAI.: Tactic: use delimiters to clearly indicate distinct parts of the input.  Accessed: 2023-09-01.  <https://platform.openai.com/docs/guides/gpt-best-practices/tactic-use-delimiters-to-clearly-indicate-distinct-parts-of-the-input>. 
  * Logan IV et al. [2022] Logan IV R, Balažević I, Wallace E, Petroni F, Singh S, Riedel S.  Cutting down on prompts and parameters: simple few-shot learning with language models.  In: Findings of the Association for Computational Linguistics: ACL 2022; 2022. p. 2824–2835. 
  * Shyr et al. [2023] Shyr C, Hu Y, Harris PA, Xu H.  Identifying and extracting rare disease phenotypes with large language models; 2023.  ArXiv:2306.12656. 
  * Reynolds and McDonell [2021] Reynolds L, McDonell K.  Prompt programming for large language models: beyond the few-shot paradigm.  In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems; 2021. p. 1–7. 
  * Liu et al. [2023] Liu P, Yuan W, Fu J, Jiang Z, Hayashi H, Neubig G.  Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing.  ACM Comput Surv. 2023 jan;55(9). 
  * Kojima et al. [2022] Kojima T, Gu SS, Reid M, Matsuo Y, Iwasawa Y.  Large language models are zero-shot reasoners.  Advances in Neural Information Processing Systems. 2022;35:22199–22213. 
  * Liu et al. [2020] Liu J, Gardner M, Cohen SB, Lapata M.  Multi-step inference for reasoning over paragraphs; 2020.  ArXiv:2004.02995. 
  * Ackley et al. [1985] Ackley DH, Hinton GE, Sejnowski TJ.  A learning algorithm for Boltzmann machines.  Cognitive Science. 1985;9(1):147–169. 
  * Ficler and Goldberg [2017] Ficler J, Goldberg Y.  Controlling linguistic style aspects in neural language generation.  In: Proceedings of the Workshop on Stylistic Variation; 2017. p. 94–104. 
  * Xu et al. [2023] Xu C, Guo D, Duan N, McAuley J.  RIGA at SemEval-2023 Task 2: NER enhanced with GPT-3.  In: Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023); 2023. p. 331–339. 
  * Liesenfeld and Dingemanse [2024] Liesenfeld A, Dingemanse M.  Rethinking open source generative AI: open washing and the EU AI Act.  In: The 2024 ACM Conference on Fairness, Accountability, and Transparency; 2024. p. 1774–1787. 
  * Wu et al. [2023] Wu S, Shen EM, Badrinath C, Ma J, Lakkaraju H.  Analyzing chain-of-thought prompting in large language models via gradient-based feature attributions; 2023.  ArXiv:2307.13339. 
  * Zhang et al. [2023] Zhang Z, Zhang A, Li M, Smola A.  Automatic chain of thought prompting in Large language models.  In: Eleventh International Conference on Learning Representations; 2023. . 
  * Lewkowycz et al. [2022] Lewkowycz A, Andreassen A, Dohan D, Dyer E, Michalewski H, Ramasesh V, et al.  Solving quantitative reasoning problems with language models.  Advances in Neural Information Processing Systems. 2022;35:3843–3857. 
  * Zhou et al. [2022] Zhou H, Nova A, Larochelle H, Courville A, Neyshabur B, Sedghi H.  Teaching algorithmic reasoning via in-context learning; 2022.  ArXiv:2211.09066. 
  * Lee et al. [2023] Lee N, Sreenivasan K, Lee JD, Lee K, Papailiopoulos D.  Teaching arithmetic to small transformers; 2023.  ArXiv:2307.03381. 
  * Wei et al. [2022] Wei J, Wang X, Schuurmans D, Bosma M, Ichter B, Xia F, et al.  Large language models perform diagnostic reasoning.  In: Eleventh International Conference on Learning Representations; 2022. . 
  * Zhang and Parkes [2023] Zhang H, Parkes DC.  Chain-of-thought reasoning is a policy improvement operator; 2023.  ArXiv:2309.08589. 
  * Cobbe et al. [2021] Cobbe K, Kosaraju V, Bavarian M, Chen M, Jun H, Kaiser L, et al.  Training verifiers to solve math word problems; 2021.  ArXiv:2110.14168. 
  * Huang et al. [2023] Huang S, Dong L, Wang W, Hao Y, Singhal S, Ma S, et al.  Language is not all you need: aligning perception with language models; 2023.  ArXiv:2302.14045. 
  * Shum et al. [2023] Shum K, Diao S, Zhang T.  Automatic prompt augmentation and selection with chain-of-thought from labeled data.  In: Bouamor H, Pino J, Bali K, editors. Findings of the Association for Computational Linguistics: EMNLP 2023. Singapore: Association for Computational Linguistics; 2023. p. 12113–12139. 
  * Del and Fishel [2023] Del M, Fishel M.  True detective: a deep abductive reasoning benchmark undoable for GPT-3 and challenging for GPT-4.  In: Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023); 2023. p. 314–322. 
  * Ouyang et al. [2022] Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, et al.  Training language models to follow instructions with human feedback.  Advances in Neural Information Processing Systems. 2022;35:27730–27744. 
  * Saparov and He [2022] Saparov A, He H.  Language models are greedy reasoners: a systematic formal analysis of chain-of-thought; 2022.  ArXiv:2210.01240. 
  * Tafjord et al. [2021] Tafjord O, Dalvi B, Clark P.  ProofWriter: generating implications, proofs, and abductive statements over natural language.  In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021; 2021. p. 3621–3634. 
  * Chowdhery et al. [2022] Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, et al.  Palm: scaling language modeling with pathways; 2022.  ArXiv:2204.02311. 
  * Fan et al. [2018] Fan A, Lewis M, Dauphin Y.  Hierarchical neural story generation.  In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); 2018. p. 889–898. 
  * Holtzman et al. [2018] Holtzman A, Buys J, Forbes M, Bosselut A, Golub D, Choi Y.  Learning to write with cooperative discriminators.  In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); 2018. p. 1638–1649. 
  * Huang et al. [2022] Huang J, Gu SS, Hou L, Wu Y, Wang X, Yu H, et al.  Large language models can self-improve; 2022.  ArXiv:2210.11610. 
  * Shum et al. [2023] Shum K, Diao S, Zhang T.  Automatic prompt augmentation and selection with chain-of-thought from labeled data; 2023.  ArXiv:2302.12822. 
  * Khalifa et al. [2023] Khalifa M, Logeswaran L, Lee M, Lee H, Wang L.  Discriminator-guided multi-step reasoning with language models; 2023.  ArXiv:2305.14934. 
  * Liu et al. [2022] Liu J, Liu A, Lu X, Welleck S, West P, Le Bras R, et al.  Generated knowledge prompting for commonsense reasoning.  In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); 2022. p. 3154–3169. 
  * Zhou et al. [2023] Zhou D, Schärli N, Hou L, Wei J, Scales N, Wang X, et al.  Least-to-most prompting enables complex reasoning in Large language models.  In: Eleventh International Conference on Learning Representations; 2023. . 
  * Gao et al. [2023] Gao L, Madaan A, Zhou S, Alon U, Liu P, Yang Y, et al.  Pal: program-aided language models.  In: International Conference on Machine Learning. PMLR; 2023. p. 10764–10799. 
  * Patel et al. [2021] Patel A, Bhattamishra S, Goyal N.  Are NLP Models really able to Solve Simple Math Word Problems?  In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics; 2021. p. 2080–2094. 
  * Yao et al. [2023] Yao S, Yu D, Zhao J, Shafran I, Griffiths TL, Cao Y, et al.  Tree of thoughts: deliberate problem solving with large language models; 2023.  ArXiv:2305.10601. 
  * Long [2023] Long J.  Large language model guided tree-of-thought; 2023.  ArXiv:2305.08291. 
  * Besta et al. [2023] Besta M, Blach N, Kubicek A, Gerstenberger R, Gianinazzi L, Gajda J, et al.  Graph of thoughts: solving elaborate problems with large language models; 2023.  ArXiv:2308.09687. 
  * Wang et al. [2023] Wang L, Ma C, Feng X, Zhang Z, Yang H, Zhang J, et al.  A survey on large language model based autonomous agents; 2023.  ArXiv:2308.11432. 
  * Khot et al. [2023] Khot T, Trivedi H, Finlayson M, Fu Y, Richardson K, Clark P, et al.  Decomposed prompting: a modular approach for solving complex tasks; 2023.  ArXiv:2210.02406. 
  * Diao et al. [2024] Diao S, Wang P, Lin Y, Zhang T.  Active prompting with chain-of-thought for large language models; 2024.  ArXiv:2302.12246. 
  * Li and Liang [2021] Li XL, Liang P.  Prefix-tuning: optimizing continuous prompts for generation.  In: Zong C, Xia F, Li W, Navigli R, editors. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Online: Association for Computational Linguistics; 2021. p. 4582–4597. 
  * Sahoo et al. [2024] Sahoo P, Singh AK, Saha S, Jain V, Mondal S, Chadha A.  A systematic survey of prompt engineering in large language models: techniques and applications; 2024.  ArXiv:2402.07927. 
  * Settles [2009] Settles B.: Active learning literature survey.  University of Wisconsin-Madison Department of Computer Sciences. 
  * Culotta and McCallum [2005] Culotta A, McCallum A.  Reducing labeling effort for structured prediction tasks.  In: Proceedings of the 20th National Conference on Artificial Intelligence - Volume 2. AAAI’05. AAAI Press; 2005. p. 746–751. 
  * White et al. [2023] White J, Fu Q, Hays S, Sandborn M, Olea C, Gilbert H, et al.  A prompt pattern catalog to enhance prompt engineering with ChatGPT; 2023.  ArXiv:2302.11382. 
  * Desmond and Brachman [2024] Desmond M, Brachman M.  Exploring prompt engineering practices in the enterprise; 2024.  ArXiv:2403.08950. 
  * Mondal et al. [2024] Mondal S, Bappon SD, Roy CK.  Enhancing user interaction in ChatGPT: characterizing and consolidating multiple prompts for issue resolution; 2024.  ArXiv:2402.04568. 
  * White et al. [2023] White J, Hays S, Fu Q, Spencer-Smith J, Schmidt DC.  ChatGPT prompt patterns for improving code quality, refactoring, requirements elicitation, and software design; 2023.  ArXiv:2303.07839. 
  * Pryzant et al. [2023] Pryzant R, Iter D, Li J, Lee Y, Zhu C, Zeng M.  Automatic prompt optimization with “gradient descent” and beam search.  In: Bouamor H, Pino J, Bali K, editors. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics; 2023. p. 7957–7968. 
  * Chen et al. [2023] Chen Y, Wen Z, Fan G, Chen Z, Wu W, Liu D, et al.  MAPO: boosting large language model performance with model-adaptive prompt optimization.  In: Bouamor H, Pino J, Bali K, editors. Findings of the Association for Computational Linguistics: EMNLP 2023. Singapore: Association for Computational Linguistics; 2023. p. 3279–3304. 
  * Cheng et al. [2024] Cheng J, Liu X, Zheng K, Ke P, Wang H, Dong Y, et al.  Black-box prompt optimization: aligning large language models without model training.  In: Ku LW, Martins A, Srikumar V, editors. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand: Association for Computational Linguistics; 2024. p. 3201–3219. 
  * Wang et al. [2023] Wang X, Li C, Wang Z, Bai F, Luo H, Zhang J, et al.  PromptAgent: strategic planning with language models enables expert-level prompt optimization; 2023.  ArXiv:2310.16427. 
  * Bach et al. [2022] Bach SH, Sanh V, Yong ZX, Webson A, Raffel C, Nayak NV, et al.  Promptsource: an integrated development environment and repository for natural language prompts; 2022.  ArXiv:2202.01279. 
  * Deng et al. [2022] Deng M, Wang J, Hsieh CP, Wang Y, Guo H, Shu T, et al.  RLPrompt: optimizing discrete text prompts with reinforcement learning.  In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing; 2022. p. 3369–3391. 
  * Awal et al. [2024] Awal R, Zhang L, Agrawal A.  Investigating prompting techniques for zero- and few-shot visual question answering; 2024.  ArXiv:2306.09996. 
  * OpenAI [2023] OpenAI.: ChatGPT plugins.  Accessed: 2023-10-15.  <https://openai.com/blog/chatgpt-plugins>. 
  * OpenAI [2024] OpenAI.: GPTs: introducing the latest in conversational AI.  Accessed: 2024-05-22.  Available from: <https://openai.com/index/introducing-gpts/>. 
  * Bisson [2023] Bisson S.: Microsoft build 2023: Microsoft extends its copilots with open standard plugins.  Accessed: 2023-05-25.  <https://www.techrepublic.com/article/microsoft-extends-copilot-with-open-standard-plugins/>. 
  * Ng [2023] Ng A.: ChatGPT prompt engineering for developers.  Accessed: 2023-07-18.  <https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/>. 
  * Roller et al. [2021] Roller S, Dinan E, Goyal N, Ju D, Williamson M, Liu Y, et al.  Recipes for building an open-domain chatbot.  In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume; 2021. p. 300–325. 
  * whatplugin..\ldotp.ai [2023] whatplugin..\ldotp.ai.: Prompt enhancer & ChatGPT plugins for AI development tools like prompt enhancer.  Accessed: 2023-09-14.  <https://www.whatplugin.ai/plugins/prompt-enhancer>. 
  * AISEO..\ldotp.ai [2023] AISEO..\ldotp.ai.: AISEO.  Accessed: 2023-8-15.  <https://aiseo.ai/>. 
  * ChatGPT for Search Engines [2023] ChatGPT for Search Engines.: Prompt perfect plugin for ChatGPT.  Accessed: 2023-10-15.  <https://chatonai.org/prompt-perfect-chatgpt-plugin>. 
  * Prompt Perfect [2023] Prompt Perfect.: Terms of service.  Accessed: 2023-09-20.  <https://promptperfect.xyz/static/terms.html>. 
  * Lee et al. [2018] Lee K, Firat O, Agarwal A, Fannjiang C, Sussillo D.  Hallucinations in neural machine translation; 2018.  Https://openreview.net/forum?id=SkxJ-309FQ. 
  * Ji et al. [2023] Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, et al.  Survey of hallucination in natural language generation.  ACM Computing Surveys. 2023;55(12):1–38. 
  * Lazaridou et al. [2022] Lazaridou A, Gribovskaya E, Stokowiec W, Grigorev N.  Internet-augmented language models through few-shot prompting for open-domain question answering; 2022.  ArXiv:2203.05115. 
  * Jiang et al. [2023] Jiang Z, Xu FF, Gao L, Sun Z, Liu Q, Dwivedi-Yu J, et al.  Active retrieval augmented generation; 2023.  ArXiv:2305.06983. 
  * Ram et al. [2023] Ram O, Levine Y, Dalmedigos I, Muhlgay D, Shashua A, Leyton-Brown K, et al.  In-context retrieval-augmented language models; 2023.  ArXiv:2302.00083. 
  * Shuster et al. [2021] Shuster K, Poff S, Chen M, Kiela D, Weston J.  Retrieval augmentation reduces hallucination in conversation; 2021.  ArXiv:2104.07567. 
  * Lewis et al. [2020] Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, et al.  Retrieval-augmented generation for knowledge-intensive NLP tasks.  Advances in Neural Information Processing Systems. 2020;33:9459–9474. 
  * Izacard and Grave [2020] Izacard G, Grave E.  Leveraging passage retrieval with generative models for open domain question answering; 2020.  ArXiv:2007.01282. 
  * Lewis et al. [2020] Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, et al.  BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics; 2020. p. 7871–7880. 
  * Raffel et al. [2020] Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, et al.  Exploring the limits of transfer learning with a unified text-to-text transformer.  Journal of Machine Learning Research. 2020;21(1):5485–5551. 
  * Roller et al. [2021] Roller S, Dinan E, Goyal N, Ju D, Williamson M, Liu Y, et al.  Recipes for building an open-domain chatbot.  In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume; 2021. p. 300–325. 
  * Dhuliawala et al. [2023] Dhuliawala S, Komeili M, Xu J, Raileanu R, Li X, Celikyilmaz A, et al.  Chain-of-verification reduces hallucination in large language models; 2023.  ArXiv:2309.11495. 
  * Li et al. [2023] Li J, Tang T, Zhao WX, Wang J, Nie JY, Wen JR.  The web can be your oyster for improving Large Language Models; 2023.  ArXiv:2305.10998. 
  * Paranjape et al. [2023] Paranjape B, Lundberg S, Singh S, Hajishirzi H, Zettlemoyer L, Ribeiro MT.  ART: automatic multi-step reasoning and tool-use for large language models; 2023.  ArXiv:2303.09014. 
  * Greyling [2023] Greyling C.: 12 prompt engineering techniques.  Available from: <https://cobusgreyling.medium.com/12-prompt-engineering-techniques-644481c857aa>. 
  * Vinija [2023] Vinija.: Prompt engineering.  Available from: <https://vinija.ai/nlp/prompt-engineering/>. 
  * Badhan [2023] Badhan M.: Advanced prompt engineering techniques.  Accessed: 2024-05-22.  Available from: <https://www.mercity.ai/blog-post/advanced-prompt-engineering-techniques>. 
  * Yao et al. [2023] Yao S, Zhao J, Yu D, Du N, Shafran I, Narasimhan K, et al.  ReAct: synergizing reasoning and acting in language models; 2023.  ArXiv:2210.03629. 
  * Li [2023] Li A.: ReAct: a new framework for prompt engineering in large language models.  Available from: <https://www.perxeive.com/blog/react-a-new-framework-for-prompt-engineering-in-large-language-models>. 
  * Roberts [2023] Roberts A.: How to ReAct to simple AI agents.  Available from: <https://arize.com/blog-course/react-agent-llm/>. 
  * Antol et al. [2015] Antol S, Agrawal A, Lu J, Mitchell M, Batra D, Zitnick CL, et al.  VQA: visual question answering.  In: Proceedings of the IEEE International Conference on Computer Vision (ICCV); 2015. p. 2425–2433. 
  * Wu et al. [2017] Wu Q, Teney D, Wang P, Shen C, Dick A, van den Hengel A.  Visual question answering: a survey of methods and datasets.  Computer Vision and Image Understanding. 2017;163:21–40.  Language in Vision. 
  * Wang et al. [2018] Wang P, Wu Q, Shen C, Dick A, van den Hengel A.  FVQA: fact-based visual question answering.  IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018;40(10):2413–2427. 
  * Kafle and Kanan [2017] Kafle K, Kanan C.  Visual question answering: datasets, algorithms, and future challenges.  Computer Vision and Image Understanding. 2017;163:3–20.  Language in Vision. 
  * Yin et al. [2024] Yin S, Fu C, Zhao S, Li K, Sun X, Xu T, et al.  A survey on multimodal large language models; 2024.  ArXiv:2306.13549. 
  * Wu et al. [2023] Wu J, Gan W, Chen Z, Wan S, Yu PS.  Multimodal large language models: a survey.  In: 2023 IEEE International Conference on Big Data; 2023. p. 2247–2256. 
  * Zhang et al. [2022] Zhang R, Wei Z, Fang R, Gao P, Li K, Dai J, et al.  Tip-adapter: training-free adaption of CLIP for few-shot classification; 2022.  ArXiv:2207.09519. 
  * Ju et al. [2022] Ju C, Han T, Zheng K, Zhang Y, Xie W.  Prompting visual-language models for efficient video understanding.  In: Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXV. Berlin, Heidelberg: Springer-Verlag; 2022. p. 105–124. 
  * Zhou et al. [2022] Zhou K, Yang J, Loy CC, Liu Z.  Learning to prompt for vision-language models.  International Journal of Computer Vision. 2022 jul;130(9):2337–2348. 
  * Jia et al. [2021] Jia C, Yang Y, Xia Y, Chen YT, Parekh Z, Pham H, et al.  Scaling up visual and vision-language representation learning with noisy text supervision.  In: Meila M, Zhang T, editors. Proceedings of the 38th International Conference on Machine Learning. vol. 139 of Proceedings of Machine Learning Research. PMLR; 2021. p. 4904–4916. 
  * Ma et al. [2023] Ma C, Liu Y, Deng J, Xie L, Dong W, Xu C.  Understanding and mitigating overfitting in prompt tuning for vision-language models; 2023.  ArXiv:2211.02219. 
  * Agnolucci et al. [2023] Agnolucci L, Baldrati A, Todino F, Becattini F, Bertini M, Bimbo AD.  ECO: ensembling context optimization for vision-language models; 2023.  ArXiv:2307.14063. 
  * Chowdhury et al. [2023] Chowdhury S, Nag S, Manocha D.  APoLLo : unified adapter and prompt learning for vision language models.  In: Bouamor H, Pino J, Bali K, editors. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics; 2023. p. 10173–10187. 
  * Zhou et al. [2022] Zhou K, Yang J, Loy CC, Liu Z.  Conditional prompt learning for vision-language models; 2022.  ArXiv:2203.05557. 
  * Ma et al. [2024] Ma S, Xie CW, Wei Y, Sun S, Fan J, Bao X, et al.  Understanding the multi-modal prompts of the pre-trained vision-language model; 2024.  ArXiv:2312.11570. 
  * Khattak et al. [2023] Khattak MU, Wasim ST, Naseer M, Khan S, Yang MH, Khan FS.  Self-regulating prompts: foundational model adaptation without forgetting; 2023.  ArXiv:2307.06948. 
  * Khattak et al. [2023] Khattak MU, Rasheed H, Maaz M, Khan S, Khan FS.  MaPLe: multi-modal prompt learning; 2023.  ArXiv:2210.03117. 
  * Gu et al. [2023] Gu J, Han Z, Chen S, Beirami A, He B, Zhang G, et al.  A systematic survey of prompt engineering on vision-language foundation models; 2023.  ArXiv:2307.12980. 
  * Shen et al. [2023] Shen L, Tan W, Zheng B, Khashabi D.  Flatness-aware prompt selection improves accuracy and sample efficiency; 2023.  ArXiv:2305.10713. 
  * Paul et al. [2023] Paul D, Ismayilzada M, Peyrard M, Borges B, Bosselut A, West R, et al.  Refiner: reasoning feedback on intermediate representations; 2023.  ArXiv:2304.01904. 
  * Ananthakrishnan et al. [2007] Ananthakrishnan R, Bhattacharyya P, Sasikumar M, Shah RM.  Some issues in automatic evaluation of English-Hindi MT: more blues for BLEU.  Icon. 2007;64. 
  * Callison-Burch et al. [2006] Callison-Burch C, Osborne M, Koehn P.  Re-evaluating the role of BLEU in machine translation research.  In: 11th Conference of the European Chapter of the Association for Computational Linguistics; 2006. p. 249–256. 
  * Stent et al. [2005] Stent A, Marge M, Singhai M.  Evaluating evaluation methods for generation in the presence of variation.  In: International Conference on Intelligent Text Processing and Computational Linguistics. Springer; 2005. p. 341–351. 
  * Adams et al. [2023] Adams G, Fabbri A, Ladhak F, Lehman E, Elhadad N.  From sparse to dense: GPT-4 summarization with chain of density prompting; 2023.  ArXiv:2309.04269. 
  * Stiennon et al. [2020] Stiennon N, Ouyang L, Wu J, Ziegler D, Lowe R, Voss C, et al.  Learning to summarize with human feedback.  Advances in Neural Information Processing Systems. 2020;33:3008–3021. 
  * Wang et al. [2023] Wang R, Wang H, Mi F, Chen Y, Xue B, Wong KF, et al.  Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting; 2023.  ArXiv:2305.13733. 
  * Lee et al. [2022] Lee M, Srivastava M, Hardy A, Thickstun J, Durmus E, Paranjape A, et al.  Evaluating human-language model interaction; 2022.  ArXiv:2212.09746. 
  * Papineni et al. [2002] Papineni K, Roukos S, Ward T, Zhu WJ.  BLEU: a method for automatic evaluation of machine translation.  In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics; 2002. p. 311–318. 
  * Chin-Yew [2004] Chin-Yew L.  ROUGE: a package for automatic evaluation of summaries.  In: Text Summarization Branches Out; 2004. p. 74–81. 
  * Banerjee and Lavie [2005] Banerjee S, Lavie A.  METEOR: an automatic metric for MT evaluation with improved correlation with human judgments.  In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization; 2005. p. 65–72. 
  * Zhang et al. [2020] Zhang T, Kishore V, Wu F, Weinberger KQ, Artzi Y.  BERTScore: evaluating text generation with BERT.  In: Ninth International Conference on Learning Representations; 2020. . 
  * Sai et al. [2022] Sai AB, Mohankumar AK, Khapra MM.  A survey of evaluation metrics used for NLG systems.  ACM Computing Surveys (CSUR). 2022;55(2):1–39. 
  * Srivastava et al. [2023] Srivastava A, Rastogi A, Rao A, Shoeb AAM, Abid A, Fisch A, et al.  Beyond the imitation game: quantifying and extrapolating the capabilities of language models; 2023.  ArXiv:2206.04615. 
  * Srivastava et al. [2022] Srivastava A, Rastogi A, Rao A, Shoeb AAM, Abid A, Fisch A, et al.  Beyond the imitation game: quantifying and extrapolating the capabilities of language models; 2022.  ArXiv:2206.04615. 
  * Chang et al. [2024] Chang Y, Wang X, Wang J, Wu Y, Yang L, Zhu K, et al.  A survey on evaluation of large language models.  ACM Transactions on Intelligent Systems and Technology. 2024;15(3):1–45. 
  * Wang et al. [2017] Wang Y, Liu X, Shi S.  Deep neural solver for math word problems.  In: Proceedings of the 2017 conference on empirical methods in natural language processing; 2017. p. 845–854. 
  * Qin et al. [2020] Qin J, Lin L, Liang X, Zhang R, Lin L.  Semantically-aligned universal tree-structured solver for math word problems; 2020.  ArXiv:2010.06823. 
  * Shi et al. [2015] Shi S, Wang Y, Lin CY, Liu X, Rui Y.  Automatically solving number word problems by semantic parsing and reasoning.  In: Proceedings of the 2015 conference on empirical methods in natural language processing; 2015. p. 1132–1142. 
  * Hosseini et al. [2014] Hosseini MJ, Hajishirzi H, Etzioni O, Kushman N.  Learning to Solve Arithmetic Word Problems with Verb Categorization.  In: Moschitti A, Pang B, Daelemans W, editors. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: Association for Computational Linguistics; 2014. p. 523–533. 
  * Roy and Roth [2017] Roy S, Roth D.  Unit dependency graph and its application to arithmetic word problem solving.  In: Proceedings of the AAAI conference on artificial intelligence. vol. 31; 2017. . 
  * Koncel-Kedziorski et al. [2016] Koncel-Kedziorski R, Roy S, Amini A, Kushman N, Hajishirzi H.  MAWPS: a math word problem repository.  In: Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: human language technologies; 2016. p. 1152–1157. 
  * Miao et al. [2021] Miao SY, Liang CC, Su KY.  A diverse corpus for evaluating and developing English math word problem solvers; 2021.  ArXiv:2106.15772. 
  * Goswami et al. [2024] Goswami M, Sanil V, Choudhry A, Srinivasan A, Udompanyawit C, Dubrawski A.  AQuA: a benchmarking tool for label quality assessment; 2024.  ArXiv:2306.09467. 
  * Amini et al. [2019] Amini A, Gabriel S, Lin P, Koncel-Kedziorski R, Choi Y, Hajishirzi H.  Mathqa: Towards interpretable math word problem solving with operation-based formalisms; 2019.  ArXiv:1905.13319. 
  * Koncel-Kedziorski et al. [2015] Koncel-Kedziorski R, Hajishirzi H, Sabharwal A, Etzioni O, Ang SD.  Parsing algebraic word problems into equations.  Transactions of the Association for Computational Linguistics. 2015;3:585–597. 
  * Roy and Roth [2016] Roy S, Roth D.  Solving general arithmetic word problems; 2016.  ArXiv:1608.01413. 
  * Hendrycks et al. [2021] Hendrycks D, Burns C, Kadavath S, Arora A, Basart S, Tang E, et al.  Measuring mathematical problem solving with the math dataset; 2021.  ArXiv:2103.03874. 
  * Lightman et al. [2023] Lightman H, Kosaraju V, Burda Y, Edwards H, Baker B, Lee T, et al.  Let’s verify step by step; 2023.  ArXiv:2305.20050. 
  * Hendrycks et al. [2020] Hendrycks D, Burns C, Basart S, Zou A, Mazeika M, Song D, et al.  Measuring massive multitask language understanding; 2020.  ArXiv:2009.03300. 
  * Thorne et al. [2018] Thorne J, Vlachos A, Christodoulopoulos C, Mittal A.  FEVER: a large-scale dataset for fact extraction and VERification; 2018.  ArXiv:1803.05355. 
  * Feng et al. [2023] Feng S, Shi W, Bai Y, Balachandran V, He T, Tsvetkov Y.  Knowledge card: filling LLMs’ knowledge gaps with plug-in specialized Language Models; 2023.  ArXiv:2305.09955. 
  * Kočiskỳ et al. [2018] Kočiskỳ T, Schwarz J, Blunsom P, Dyer C, Hermann KM, Melis G, et al.  The narrativeqa reading comprehension challenge.  Transactions of the Association for Computational Linguistics. 2018;6:317–328. 
  * Pang et al. [2021] Pang RY, Parrish A, Joshi N, Nangia N, Phang J, Chen A, et al.  QuALITY: question answering with long input texts, yes!  In: North American Chapter of the Association for Computational Linguistics; 2021. p. 5336–5358. 
  * Talmor et al. [2018] Talmor A, Herzig J, Lourie N, Berant J.  Commonsenseqa: a question answering challenge targeting commonsense knowledge; 2018.  ArXiv:1811.00937. 
  * Talmor et al. [2019] Talmor A, Herzig J, Lourie N, Berant J.  CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge.  In: Burstein J, Doran C, Solorio T, editors. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, Minnesota: Association for Computational Linguistics; 2019. p. 4149–4158. 
  * Speer et al. [2017] Speer R, Chin J, Havasi C.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In: Proceedings of the AAAI conference on artificial intelligence. vol. 31; 2017. . 
  * Yang et al. [2018] Yang Z, Qi P, Zhang S, Bengio Y, Cohen WW, Salakhutdinov R, et al.  HotpotQA: a dataset for diverse, explainable multi-hop question answering.  In: Conference on Empirical Methods in Natural Language Processing; 2018. p. 2369–2380. 
  * Clark et al. [2018] Clark P, Cowhey I, Etzioni O, Khot T, Sabharwal A, Schoenick C, et al.  Think you have solved question answering? try arc, the ai2 reasoning challenge; 2018.  ArXiv:1803.05457. 
  * Huang et al. [2021] Huang L, Cao S, Parulian N, Ji H, Wang L.  Efficient attentions for long document summarization; 2021.  ArXiv:2104.02112. 
  * Voorhees and Tice [2000] Voorhees EM, Tice DM.  Building a question answering test collection.  In: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval; 2000. p. 200–207. 
  * Socher et al. [2013] Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng AY, et al.  Recursive deep models for semantic compositionality over a sentiment treebank.  In: Proceedings of the 2013 conference on empirical methods in natural language processing; 2013. p. 1631–1642. 
  * Chen et al. [2021] Chen M, Chu Z, Wiseman S, Gimpel K.  Summscreen: a dataset for abstractive screenplay summarization; 2021.  ArXiv:2104.07091. 
  * Zhang et al. [2015] Zhang X, Zhao J, LeCun Y.  Character-level convolutional networks for text classification.  Advances in neural information processing systems. 2015;28. 
  * Zhang et al. [2023] Zhang W, Deng Y, Liu B, Pan SJ, Bing L.  Sentiment analysis in the era of large language models: a reality check; 2023.  ArXiv:2305.15005. 
  * Hu and Liu [2004] Hu M, Liu B.  Mining and summarizing customer reviews.  In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining; 2004. p. 168–177. 
  * Pang and Lee [2005] Pang B, Lee L.  Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales; 2005.  Cs/0506075. 
  * Tang et al. [2023] Tang L, Peng Y, Wang Y, Ding Y, Durrett G, Rousseau JF.  Less likely brainstorming: Using language models to generate alternative hypotheses.  In: Proceedings of the conference. Association for Computational Linguistics. Meeting. vol. 2023. NIH Public Access; 2023. p. 12532. 
  * Raunak et al. [2022] Raunak V, Post M, Menezes A.  SALTED: a framework for SAlient long-tail translation error detection; 2022.  ArXiv:2205.09988. 
  * Yu et al. [2016] Yu L, Poirson P, Yang S, Berg AC, Berg TL.  Modeling context in referring expressions.  In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14. Springer; 2016. p. 69–85. 
  * Mao et al. [2016] Mao J, Huang J, Toshev A, Camburu O, Yuille AL, Murphy K.  Generation and comprehension of unambiguous object descriptions.  In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 11–20. 
  * Jain et al. [2023] Jain N, Saifullah K, Wen Y, Kirchenbauer J, Shu M, Saha A, et al.  Bring your own data! Self-supervised evaluation for large language models; 2023.  ArXiv:2306.13651. 
  * Wang et al. [2023] Wang Y, Yu Z, Zeng Z, Yang L, Wang C, Chen H, et al.  PandaLM: an automatic evaluation benchmark for LLM instruction tuning optimization; 2023.  ArXiv:2306.05087. 
  * Lin and Chen [2023] Lin YT, Chen YN.  LLM-eval: unified multi-dimensional automatic evaluation for open-domain conversations with large language models; 2023.  ArXiv:2305.13711. 
  * Dehghani et al. [2021] Dehghani M, Tay Y, Gritsenko AA, Zhao Z, Houlsby N, Diaz F, et al.  The benchmark lottery; 2021.  ArXiv:2107.07002. 
  * Kiela et al. [2021] Kiela D, Bartolo M, Nie Y, Kaushik D, Geiger A, Wu Z, et al.  Dynabench: rethinking benchmarking in NLP; 2021.  ArXiv:2104.14337. 
  * Xu et al. [2023] Xu W, Banburski-Fahey A, Jojic N.  Reprompting: automated chain-of-thought prompt inference through gibbs sampling; 2023.  ArXiv:2305.09993. 
  * Wang et al. [2023] Wang L, Xu W, Lan Y, Hu Z, Lan Y, Lee RKW, et al.  Plan-and-solve prompting: improving zero-shot chain-of-thought reasoning by large language models; 2023.  ArXiv:2305.04091. 
  * Mishra et al. [2021] Mishra S, Khashabi D, Baral C, Hajishirzi H.  Cross-task generalization via natural language crowdsourcing instructions; 2021.  ArXiv:2104.08773. 
  * Pryzant et al. [2023] Pryzant R, Iter D, Li J, Lee YT, Zhu C, Zeng M.  Automatic prompt optimization with ”gradient descent” and beam search; 2023.  ArXiv:2305.03495. 
  * Zhou et al. [2022] Zhou Y, Muresanu AI, Han Z, Paster K, Pitis S, Chan H, et al.  Large Language Models are human-level prompt engineers.  In: Eleventh International Conference on Learning Representations; 2022. . 
  * Chen et al. [2023] Chen H, Pasunuru R, Weston J, Celikyilmaz A.  Walking down the memory maze: beyond context limit through interactive reading; 2023.  ArXiv:2310.05029. 
  * Chevalier et al. [2023] Chevalier A, Wettig A, Ajith A, Chen D.  Adapting language models to compress contexts; 2023.  ArXiv:2305.14788. 
  * Bulatov et al. [2023] Bulatov A, Kuratov Y, Kapushev Y, Burtsev MS.  Scaling transformer to 1m tokens and beyond with rmt; 2023.  ArXiv:2304.11062. 
  * Xu et al. [2021] Xu J, Szlam A, Weston J.  Beyond goldfish memory: Long-term open-domain conversation; 2021.  ArXiv:2107.07567. 
  * Wu et al. [2022] Wu Y, Rabe MN, Hutchins D, Szegedy C.  Memorizing transformers; 2022.  ArXiv:2203.08913. 
  * Izacard and Grave [2020] Izacard G, Grave E.  Leveraging passage retrieval with generative models for open domain question answering; 2020.  ArXiv:2007.01282. 
  * Guo et al. [2023] Guo Q, Wang R, Guo J, Li B, Song K, Tan X, et al.  Connecting large language models with evolutionary algorithms yields powerful prompt optimizers; 2023.  ArXiv:2309.08532. 
  * Sridhar et al. [2023] Sridhar A, Lo R, Xu FF, Zhu H, Zhou S.  Hierarchical prompting assists large language model on web navigation; 2023.  ArXiv:2305.14257. 
  * Yao et al. [2022] Yao S, Chen H, Yang J, Narasimhan K.  Webshop: towards scalable real-world web interaction with grounded language agents.  Advances in Neural Information Processing Systems. 2022;35:20744–20757. 
  * Zhang et al. [2023] Zhang C, Xiao J, Chen L, Shao J, Chen L.  TreePrompt: Learning to Compose Tree Prompts for Explainable Visual Grounding; 2023.  ArXiv:2305.11497. 
  * Colangeli et al. [2023] Colangeli M, Di Francesco A, Rondoni L.  Finite reservoirs and irreversibility corrections to Hamiltonian systems statistics; 2023.  ArXiv:2305.14922. 
  * Jiang et al. [2023] Jiang H, Wu Q, Lin CY, Yang Y, Qiu L.  Llmlingua: Compressing prompts for accelerated inference of large language models; 2023.  ArXiv:2310.05736. 
  * Ning et al. [2023] Ning X, Lin Z, Zhou Z, Yang H, Wang Y.  Skeleton-of-thought: Large language models can do parallel decoding; 2023.  ArXiv:2307.15337. 
  * Hu et al. [2023] Hu H, Lu H, Zhang H, Song YZ, Lam W, Zhang Y.  Chain-of-symbol prompting elicits planning in Large Langauge Models; 2023.  ArXiv:2305.10276. 
  * Feng et al. [2023] Feng S, Shi W, Bai Y, Balachandran V, He T, Tsvetkov Y.  Knowledge Card: Filling LLMs’ Knowledge Gaps with Plug-in Specialized Language Models; 2023.  ArXiv:2305.09955. 
  * Krishna et al. [2024] Krishna S, Ma J, Slack D, Ghandeharioun A, Singh S, Lakkaraju H.  Post hoc explanations of language models can improve language models.  Advances in Neural Information Processing Systems. 2024;36. 
  * Sun et al. [2023] Sun S, Liu Y, Wang S, Zhu C, Iyyer M.  Pearl: Prompting large language models to plan and execute actions over long documents; 2023.  ArXiv:2305.14564. 
  * Chen et al. [2022] Chen W, Ma X, Wang X, Cohen WW.  Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks; 2022.  ArXiv:2211.12588. 
  * Press et al. [2022] Press O, Zhang M, Min S, Schmidt L, Smith NA, Lewis M.  Measuring and narrowing the compositionality gap in language models; 2022.  ArXiv:2210.03350. 
  * Schick et al. [2024] Schick T, Dwivedi-Yu J, Dessì R, Raileanu R, Lomeli M, Hambro E, et al.  Toolformer: Language models can teach themselves to use tools.  Advances in Neural Information Processing Systems. 2024;36. 
  * Zhou et al. [2023] Zhou Y, Muresanu AI, Han Z, Paster K, Pitis S, Chan H, et al.  Large Language Models Are Human-Level Prompt Engineers; 2023.  ArXiv:2211.01910. 
  * Ajith et al. [2023] Ajith A, Pan C, Xia M, Deshpande A, Narasimhan K.  InstructEval: systematic evaluation of instruction selection methods; 2023.  ArXiv:2307.00259. 
  * Tang et al. [2023] Tang J, Zhou X, Wan X, Daley M, Bai Z.  ML4STEM professional development program: enriching K-12 STEM teaching with machine learning.  International Journal of Artificial Intelligence in Education. 2023;33(1):185–224. 
  * Xie et al. [2020] Xie Q, Dai Z, Hovy E, Luong MT, Le QV.  Unsupervised data augmentation for consistency training.  In: Advances in neural information processing systems. vol. 33; 2020. p. 6256–6268. 
  * Ariely et al. [2023] Ariely M, Nazaretsky T, Alexandron G.  Machine learning and Hebrew NLP for automated assessment of open-ended questions in biology.  International Journal of Artificial Intelligence in Education. 2023;33(1):1–34. 
  * Nilsson and Tuvstedt [2023] Nilsson F, Tuvstedt J.  GPT-4 as an automatic grader: the accuracy of grades set by GPT-4 on introductory programming assignments [Bachelor Thesis].  KTH Royal Institute of Technology; 2023. 
  * Schneider et al. [2023] Schneider J, Richner R, Riser M.  Towards trustworthy autograding of short, multi-lingual, multi-type answers.  International Journal of Artificial Intelligence in Education. 2023;33(1):88–118. 
  * Yang et al. [2022] Yang K, Tian Y, Peng N, Klein D.  Re3: generating longer stories with recursive reprompting and revision.  In: Goldberg Y, Kozareva Z, Zhang Y, editors. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics; 2022. p. 4393–4479. 
  * Yang et al. [2023] Yang K, Klein D, Peng N, Tian Y.  DOC: improving long story coherence with detailed outline control.  In: Rogers A, Boyd-Graber J, Okazaki N, editors. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics; 2023. p. 3378–3465. 
  * Yang and Klein [2021] Yang K, Klein D.  FUDGE: controlled text generation with future discriminators.  In: Toutanova K, Rumshisky A, Zettlemoyer L, Hakkani-Tur D, Beltagy I, Bethard S, et al., editors. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics; 2021. p. 3511–3535. 
  * Elgohary et al. [2020] Elgohary A, Hosseini S, Awadallah AH.  Speak to your parser: interactive text-to-SQL with natural language feedback.  In: Annual Meeting of the Association for Computational Linguistics; 2020. p. 2065–2077. 
  * Nijkamp et al. [2022] Nijkamp E, Pang B, Hayashi H, Tu L, Wang H, Zhou Y, et al.  Codegen: an open large language model for code with multi-turn program synthesis; 2022.  ArXiv:2203.13474. 
  * Shrivastava et al. [2023] Shrivastava D, Larochelle H, Tarlow D.  Repository-level prompt generation for large language models of code.  In: International Conference on Machine Learning; 2023. p. 31693–31715. 
  * Shwartz et al. [2020] Shwartz V, West P, Bras RL, Bhagavatula C, Choi Y.  Unsupervised commonsense question answering with self-talk; 2020.  ArXiv:2004.05483. 
  * Uesato et al. [2022] Uesato J, Kushman N, Kumar R, Song F, Siegel N, Wang L, et al.  Solving math word problems with process-and outcome-based feedback; 2022.  ArXiv:2211.14275. 
  * Roy and Roth [2015] Roy S, Roth D.  Solving general arithmetic word problems.  In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing; 2015. p. 1743–1752. 
  * Li et al. [2023] Li Y, Lin Z, Zhang S, Fu Q, Chen B, Lou JG, et al.  Making language models better reasoners with step-aware verifier.  In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics; 2023. p. 5315–5333. 
  * Ding et al. [2022] Ding B, Qin C, Liu L, Bing L, Joty S, Li B.  Is GPT-3 a good data annotator?; 2022.  ArXiv:2212.10450. 
  * Yoo et al. [2021] Yoo KM, Park D, Kang J, Lee SW, Park W.  GPT3Mix: leveraging large-scale language models for text augmentation.  In: Findings of the Association for Computational Linguistics: EMNLP 2021; 2021. p. 2225–2239. 
  * Liu et al. [2024] Liu Y, Deng G, Xu Z, Li Y, Zheng Y, Zhang Y, et al.  Jailbreaking ChatGPT via prompt engineering: an empirical study; 2024.  ArXiv:2305.13860. 
  * Yao et al. [2024] Yao Y, Duan J, Xu K, Cai Y, Sun Z, Zhang Y.  A survey on Large Language Model (LLM) security and privacy: the good, the bad, and the ugly.  High-Confidence Computing. 2024 jun;4(2):100211. 
  * Rawat [2023] Rawat P.: AI at risk: OWASP top 10 critical vulnerabilities for large language models (LLMs).  Available from: <https://www.infosectrain.com/blog/ai-at-risk-owasp-top-10-critical-vulnerabilities-for-large-language-models-llms/>. 
  * Perez and Ribeiro [2022] Perez F, Ribeiro I.  Ignore previous prompt: attack techniques for language models; 2022.  ArXiv:2211.09527. 
  * Ren et al. [2020] Ren K, Zheng T, Qin Z, Liu X.  Adversarial attacks and defenses in deep learning.  Engineering. 2020;6(3):346–360. 
  * Yin et al. [2024] Yin Z, Ye M, Zhang T, Du T, Zhu J, Liu H, et al.  VLATTACK: multimodal adversarial attacks on vision-language tasks via pre-trained models; 2024.  ArXiv:2310.04655. 
  * Goodfellow et al. [2015] Goodfellow IJ, Shlens J, Szegedy C.  Explaining and harnessing adversarial examples; 2015.  ArXiv:1412.6572. 
  * Deng et al. [2009] Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L.  ImageNet: a large-scale hierarchical image database.  In: 2009 IEEE Conference on Computer Vision and Pattern Recognition; 2009. p. 248–255. 
  * Wang et al. [2023] Wang J, Liu Z, Park KH, Jiang Z, Zheng Z, Wu Z, et al.  Adversarial demonstration attacks on large language models; 2023.  ArXiv:2305.14950. 
  * Kolter [2023] Kolter Z.: Discrete optimization for adversarial attacks on large language models.  Available from: <https://orc.mit.edu/events/discrete-optimization-adversarial-attacks-large-language-models>. 
  * Shayegani et al. [2023] Shayegani E, Mamun MAA, Fu Y, Zaree P, Dong Y, Abu-Ghazaleh N.  Survey of vulnerabilities in large language models revealed by adversarial attacks; 2023.  ArXiv:2310.10844. 
  * Selvakkumar et al. [2021] Selvakkumar A, Pal S, Jadidi Z.  Addressing adversarial machine learning attacks in smart healthcare perspectives; 2021.  ArXiv:2112.08862. 
  * Bagdasaryan et al. [2020] Bagdasaryan E, Veit A, Hua Y, Estrin D, Shmatikov V.  How to backdoor federated learning.  In: Chiappa S, Calandra R, editors. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics. vol. 108 of Proceedings of Machine Learning Research. PMLR; 2020. p. 2938–2948. 
  * Steinhardt et al. [2017] Steinhardt J, Koh PW, Liang P.  Certified defenses for data poisoning attacks; 2017.  ArXiv:1706.03691. 
  * Yang et al. [2024] Yang H, Xiang K, Ge M, Li H, Lu R, Yu S.  A comprehensive overview of backdoor attacks in large language models within communication networks.  IEEE Network. 2024;p. 1–1. 
  * Chen et al. [2017] Chen X, Liu C, Li B, Lu K, Song D.  Targeted backdoor attacks on deep learning systems using data poisoning; 2017.  ArXiv:1712.05526. 
  * Shamshiri and Sohn [2024] Shamshiri S, Sohn I.  Defense method challenges against backdoor attacks in neural networks.  In: 2024 International Conference on Artificial Intelligence in Information and Communication; 2024. p. 396–400. 
  * Holland et al. [2024] Holland R, Pal S, Pan L, Zhang LY.  Backdoor attacks and generative model fairness: current trends and future research directions.  In: 2024 16th International Conference on COMmunication Systems & NETworkS; 2024. p. 31–36. 
  * Khan and Sharma [2024] Khan A, Sharma I.  AI-powered detection and mitigation of backdoor attacks on databases server.  In: 2024 2nd International Conference on Intelligent Data Communication Technologies and Internet of Things; 2024. p. 374–379. 
  * Ooi [2024] Ooi YX.  Evaluation of backdoor attacks and defenses to deep neural networks [Master’s thesis].  Nanyang Technological University; 2024. 
  * Li et al. [2024] Li Y, Li T, Chen K, Zhang J, Liu S, Wang W, et al.  BadEdit: backdooring large language models by model editing; 2024.  ArXiv:2403.13355. 
  * Gu et al. [2019] Gu T, Dolan-Gavitt B, Garg S.  BadNets: identifying vulnerabilities in the machine learning model supply chain; 2019.  ArXiv:1708.06733. 
  * Liu et al. [2020] Liu Y, Ma X, Bailey J, Lu F.  Reflection backdoor: a natural backdoor attack on deep neural networks.  In: Computer Vision – ECCV 2020. Berlin, Heidelberg: Springer-Verlag; 2020. p. 182–199. 
  * Zhao et al. [2023] Zhao S, Wen J, Luu A, Zhao J, Fu J.  Prompt as triggers for backdoor attack: examining the vulnerability in language models.  In: Bouamor H, Pino J, Bali K, editors. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics; 2023. p. 12303–12317. 
  * Chow et al. [2024] Chow KH, Wei W, Yu L.  Imperio: language-guided backdoor attacks for arbitrary model control; 2024.  ArXiv:2401.01085. 
  * Abdali et al. [2024] Abdali S, Anarfi R, Barberan C, He J.  Securing large language models: threats, vulnerabilities and responsible practices; 2024.  ArXiv:2403.12503. 
  * Kosch and Feger [2024] Kosch T, Feger S.  Risk or chance? Large language models and reproducibility in human-computer interaction research; 2024.  ArXiv:2404.15782. 
  * Liu and Hu [2024] Liu FW, Hu C.  Exploring vulnerabilities and protections in large language models: a survey; 2024.  ArXiv:2406.00240. 
  * Zhan et al. [2024] Zhan Q, Liang Z, Ying Z, Kang D.  InjecAgent: benchmarking indirect prompt injections in tool-integrated large language model agents; 2024.  ArXiv:2403.02691. 
  * Chen et al. [2024] Chen K, Wang Z, Mi B, Liu W, Wang S, Ren X, et al.  Machine unlearning in large language models; 2024.  ArXiv:2404.16481. 
  * Wang et al. [2024] Wang H, Li H, Huang M, Sha L.  From noise to clarity: unraveling the adversarial suffix of large language model attacks via translation of text embeddings; 2024.  ArXiv:2402.16006. 
  * Gao [2023] Gao A.  Prompt engineering for large language models; 2023.  SSRN:4504303. 
  * Learn prompting [2024] Learn prompting.: Introduction to prompt hacking.  Available from: <https://learnprompting.org/docs/prompt_hacking/intro>. 
  * Schulhoff et al. [2023] Schulhoff S, Pinto J, Khan A, Bouchard LF, Si C, Anati S, et al.  Ignore this title and HackAPrompt: exposing systemic vulnerabilities of LLMs through a global prompt hacking competition.  In: Bouamor H, Pino J, Bali K, editors. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics; 2023. p. 4945–4977. 
  * Allouin [2023] Allouin A.: Understanding LLM prompt hacking and attacks.  Available from: <https://medium.com/@alexandre.allouin/understanding-llm-prompt-hacking-and-attacks-8781c313a25b>. 
  * Karande [2024] Karande C.: OWASP LLM prompt hacking.  Available from: <https://owasp.org/www-project-llm-prompt-hacking/>. 
  * Teneo AI [2024] Teneo AI.: Navigating the challenges of prompt hacking in 2024.  Available from: <https://www.teneo.ai/blog/navigating-the-challenges-of-prompt-hacking-in-2024>. 
  * Krishna et al. [2020] Krishna K, Tomar GS, Parikh AP, Papernot N, Iyyer M.  Thieves on sesame street! Model extraction of BERT-based APIs.  In: Ninth International Conference on Learning Representations; 2020. . 
  * Papernot et al. [2018] Papernot N, McDaniel P, Sinha A, Wellman MP.  SoK: security and privacy in machine learning.  In: 2018 IEEE European Symposium on Security and Privacy (EuroS&P); 2018. p. 399–414. 
  * Tramèr et al. [2016] Tramèr F, Zhang F, Juels A, Reiter MK, Ristenpart T.  Stealing machine learning models via prediction APIs.  In: Proceedings of the 25th USENIX Conference on Security Symposium. SEC’16. USA: USENIX Association; 2016. p. 601–618. 
  * Shen et al. [2023] Shen X, Qu Y, Backes M, Zhang Y.  Prompt stealing attacks against text-to-image generation models; 2023.  ArXiv:2302.09923. 
  * Zhang [2024] Zhang S.  Defending against model extraction attacks via watermark-based method with knowledge distillation [Master’s thesis].  Nanyang Technological University; 2024. 
  * Naseh et al. [2023] Naseh A, Krishna K, Iyyer M, Houmansadr A.  Stealing the decoding algorithms of language models.  In: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. CCS ’23. New York, NY, USA: Association for Computing Machinery; 2023. p. 1835–1849. 
  * Carlini et al. [2024] Carlini N, Paleka D, Dvijotham KD, Steinke T, Hayase J, Cooper AF, et al.  Stealing part of a production language model; 2024.  ArXiv:2403.06634. 
  * Sha and Zhang [2024] Sha Z, Zhang Y.  Prompt stealing attacks against large language models; 2024.  ArXiv:2402.12959. 
  * Yang et al. [2024] Yang Y, Li C, Jiang Y, Chen X, Wang H, Zhang X, et al.  PRSA: PRompt Stealing Attacks against large language models; 2024.  ArXiv:2402.19200. 
  * Hu et al. [2022] Hu H, Salcic Z, Sun L, Dobbie G, Yu PS, Zhang X.  Membership inference attacks on machine learning: a survey; 2022.  ArXiv:2103.07853. 
  * Kurakin et al. [2017] Kurakin A, Goodfellow I, Bengio S.  Adversarial machine learning at scale; 2017.  ArXiv:1611.01236. 
  * Bai et al. [2021] Bai T, Luo J, Zhao J, Wen B, Wang Q.  Recent advances in adversarial training for adversarial robustness.  In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization; 2021. p. 4312–4321.  Survey Track. 
  * Eric [2022] Eric M.: A complete introduction to prompt engineering for large language models.  Available from: <https://www.mihaileric.com/posts/a-complete-introduction-to-prompt-engineering/>. 
  * Linardatos et al. [2020] Linardatos P, Papastefanopoulos V, Kotsiantis S.  Explainable AI: a review of machine learning interpretability methods.  Entropy. 2020;23(1):18. 
  * Recht et al. [2011] Recht B, Re C, Wright S, Niu F.  Hogwild!: a lock-free approach to parallelizing stochastic gradient descent.  In: Advances in neural information processing systems. vol. 24; 2011. . 
  * Melnychuk et al. [2022] Melnychuk V, Frauen D, Feuerriegel S.  Causal transformer for estimating counterfactual outcomes.  In: International Conference on Machine Learning; 2022. p. 15293–15329. 
  * Novakovsky et al. [2023] Novakovsky G, Dexter N, Libbrecht MW, Wasserman WW, Mostafavi S.  Obtaining genetics insights from deep learning via explainable artificial intelligence.  Nature Reviews Genetics. 2023;24(2):125–137. 
  * Bertucci et al. [2022] Bertucci L, Brière M, Fliche O, Mikael J, Szpruch L.  Deep learning in finance: from implementation to regulation; 2022.  SSRN:4080171. 
  * Maple et al. [2023] Maple C, Szpruch L, Epiphaniou G, Staykova K, Singh S, Penwarden W, et al.  The AI revolution: opportunities and challenges for the finance sector; 2023.  ArXiv:2308.16538. 
  * Amann et al. [2020] Amann J, Blasimme A, Vayena E, Frey D, Madai VI.  Explainability for artificial intelligence in healthcare: a multidisciplinary perspective.  BMC Medical Informatics and Decision Making. 2020;20(1):1–9. 
  * Rajpurkar et al. [2022] Rajpurkar P, Chen E, Banerjee O, Topol EJ.  AI in health and medicine.  Nature Medicine. 2022;28(1):31–38. 
  * Öztürk [2021] Öztürk D.  What does artificial intelligence mean for organizations? A systematic review of organization studies research and a way forward.  The Impact of Artificial Intelligence on Governance, Economics and Finance, Volume I. 2021;p. 265–289. 
  * Seeamber and Badea [2023] Seeamber R, Badea C.  If our aim is to build morality into an artificial agent, how might we begin to go about doing so?  IEEE Intelligent Systems. 2023;. 





