Generative AI Custom Evaluation in SAP AI Core
The concept name was not provided, but based on the sources, this appears to relate to Custom Evaluation / LLM-as-a-Judge in SAP AI Core. This is a framework within SAP AI Core that allows developers to define and run custom evaluation metrics for generative AI outputs, including grounding and prompt optimization scenarios. Developers use it to assess the quality of large language model responses by configuring custom judges — including other LLMs acting as evaluators — against criteria such as tool-calling accuracy or retrieval-augmented grounding. It supports both quickstart workflows and comprehensive evaluation pipelines, enabling iterative improvement of prompts and model behavior within the SAP AI Core environment.
Tutorials that teach this
Prerequisites
- Concept SAP AI Launchpad Usage The concept name is missing or undefined, and the provided sources do not contain enough specific, attributable information to write a grounded definition for an unnamed concept. Please provide a valid concept name so a precise, source-backed definition can be written.
- Concept AWS service integration (Amazon S3 and Amazon Rekognition) The concept name was not provided, so a precise definition cannot be generated. The available source describes accessing data from Amazon S3 in SAP HANA Cloud, SAP HANA Database, but without a defined concept to document, no accurate or grounded prose definition can be written. Please supply the concept name so a correct definition can be produced.
- Concept Prompt Engineering The concept name was not provided, so a precise definition cannot be written. Please supply the concept name so it can be accurately defined based on the available sources covering topics such as prompting LLMs in the generative AI hub in SAP AI Core and using multimodal inputs with GPT-4o for image recognition on SAP AI Core.
- Concept Python SDK Installation and Configuration The concept to be defined is not specified (received as "undefined"), and the provided sources cover distinct SAP SDK and tooling setup tutorials without a shared concept to summarize. A meaningful, source-grounded definition cannot be produced without a valid concept name.
- Concept SAP AI Core Service Setup The concept name was not provided, so a precise definition cannot be constructed from the available sources. Based on the sources, these materials collectively cover building and integrating [Generative AI applications](https://architecture.learning.sap.com/docs/golden-path/ai-golden-path/2-build-and-deliver/3-genai-applications/readme) on SAP Business Technology Platform using SAP AI Core — a managed service that developers use to train models, run AI workloads, and connect to large language models via orchestration. Developers leverage it alongside [SAP BTP service instances and keys](https://help.sap.com/docs/btp/sap-business-technology-platform/5b35ee98403045309bb0f8c7f0c79365?locale=en-US&state=PRODUCTION&version=Cloud) to authenticate and consume AI capabilities such as prompt optimization, grounding, multimodal responses, and custom evaluation within their applications.
- Concept Prompt Registry Management in SAP Generative AI Hub The concept name provided is "undefined," and the supplied sources do not contain enough grounded information to produce a reliable, accurate definition for a specific SAP developer concept. Please provide a valid concept name and relevant source snippets so a precise definition can be written.
- Concept Generative AI Hub setup and onboarding The concept name provided is "undefined," and the supplied sources do not contain enough grounded information to produce a reliable, accurate definition for a specific SAP developer concept. Please provide a valid concept name and relevant source snippets so a precise definition can be written.
- Concept Cloud Object Storage Credentials and Secrets Management The concept provided is **undefined** — no concept name was supplied, and the available source snippets (covering Azure Data Bucket setup with AI Core and SAP HANA Database Explorer data export/import) do not contain sufficient shared context to infer a single, specific SAP developer concept to define. Please provide a valid concept name so an accurate, source-grounded definition can be written.
- Concept Grounding Large Language Models (LLMs) with Enterprise Data The concept name was not provided, making it impossible to write a meaningful or accurate reference definition grounded in the supplied sources. Please provide the specific concept (for example, "Grounding Evaluation," "SAP AI Core," or another term from the tutorial) so a precise definition can be written.
- Concept Retrieval Augmented Generation (RAG) The concept name was not provided (marked as "undefined"), and the available source snippets do not contain enough detail to identify or define a specific SAP developer concept with confidence. Based on the sources, the closest identifiable concept is **Retrieval Augmented Generation (RAG)**, which is a technique used to gain more control over the prompting results of a Large Language Model (LLM) by leveraging your own data. Developers use it within [Generative AI on SAP BTP](https://architecture.learning.sap.com/docs/ref-arch/RA0005/readme) to ground LLM responses in enterprise-specific content, for example through [HANA vector search or SAP AI Core orchestration](https://architecture.learning.sap.com/docs/ref-arch/RA0005/3-retrieval-augmented-generation/readme).
- Concept SAP Generative AI Hub The concept name was not provided, so a best-effort definition is derived from the available sources covering SAP's AI development landscape. [SAP's AI Golden Path](https://architecture.learning.sap.com/docs/golden-path/ai-golden-path/readme) is a curated starting point for developing AI applications across the SAP ecosystem, offering recommendations and guidance for building and delivering AI-powered solutions. Developers use it to navigate capabilities such as [Generative AI application integration, embeddings and semantic search](https://architecture.learning.sap.com/docs/ref-arch/RA0005/2-semantic-search/readme), predictive and tabular AI, and document processing. It encompasses patterns for leveraging Foundation Models and Large Language Models (LLMs) through prompting, as well as frameworks like Intelligent Scenario Lifecycle Management (ISLM) for standardized AI lifecycle management.
- Concept SAP BTP Subaccount and Entitlement Setup The provided sources do not contain sufficient information to define this concept. The concept name is "undefined" and none of the source snippets supply grounded content that could be used to write an accurate, sourced definition.
- Concept Prompt Template Engineering The concept name was not provided (received "undefined"), so a meaningful definition cannot be written. Please supply a valid SAP developer concept name so that a grounded, accurate definition can be produced from the available sources.
- Concept SAP AI Core Object Store Secret Configuration The concept name provided is undefined and no valid concept was supplied for definition. Additionally, the provided source snippets contain only tutorial titles without any substantive content to ground a definition. Please provide a valid SAP developer concept name and relevant source content so an accurate definition can be written.
- Concept Jupyter Notebook ML Workflows The concept to be defined is not specified. Based on the provided sources, which cover topics such as SAP HANA Cloud graph workspaces, machine learning with AutoML, Service Ticket Intelligence, SAP Signavio process optimization, and Python connectivity, a meaningful definition cannot be accurately constructed without knowing the specific concept in question. Please provide the concept name so a precise, source-grounded definition can be written.
- Concept Cloud Object Store Integration for ML Data The provided sources do not contain enough information to define a specific SAP developer concept, as no concept name was supplied and the source snippets consist only of tutorial titles without descriptive content. A meaningful, source-grounded definition cannot be written without at least a named concept or substantive excerpt from the referenced materials.
- Concept SAP BTP Fundamentals and Platform Navigation
- Concept SAP AI Core Orchestration Service The concept name was not provided (marked as "undefined"), and the supplied source snippets contain only tutorial titles without any descriptive body text or factual claims that could be used to construct an accurate, grounded definition. There is insufficient source material to write a valid 2–4 sentence prose definition for an unknown concept.