feat: [Orchestration] Support for Orchestration config persistence #697

Jonas-Isr · 2025-12-23T08:17:00Z

Context

https://github.com/orgs/SAP/projects/62/views/7?pane=issue&itemId=138596334&issue=SAP%7Cai-sdk-java-backlog%7C343

The PR enables the management of Orchestration Configs in prompt registry (create, delete, list via generated code and new client) as well as convenience to use these Orchestration Configs in the OrchestrationClient.

Sample of new convenience:

// create OrchestrationPrompt (for placeholder values and history) with already existing convenience
List<Message> history = List.of(new SystemMessage("System Message"));
OrchestrationPrompt prompt = new OrchestrationPrompt(Map.of("placeholder", "value")).messageHistory(history);

// create (newly added) OrchestrationConfigReference to refer to saved Orchestration config
OrchestrationConfigReference reference =
    OrchestrationConfigReference.fromScenario("scenario").name("name").version("0.0.1");

// use OrchestrationClient todo chat completion with the referenced config
OrchestrationChatResponse response = client.executeRequestFromReference(prompt, reference);

Feature scope:

add new OchestrationConfigClient in prompt registry
- unit and e2e tests for generated code and new client in prompt registry
convenience for use with OrchestrationClient
- unit and e2e tests for new convenience

Definition of Done

Functionality scope stated & covered
Tests cover the scope above
Error handling created / updated & covered by the tests above
~~Aligned changes with the JavaScript SDK~~
Documentation updated
Release notes updated

# Conflicts: # docs/release_notes.md

Jonas-Isr · 2025-12-23T08:26:29Z

orchestration/src/main/java/com/sap/ai/sdk/orchestration/ConfigToRequestTransformer.java

+      if (!prompt.getMessages().isEmpty()) {
+        log.debug(
+            "Messages in prompts are ignored when using Orchestration configs via reference. Change the Orchestration config instead.");
+      }


This could also throw instead of just logging. Was not sure which is better.

Can you give a code sample that triggers this warning?

Will be triggered e.g. by

var reference = OrchestrationConfigReference.fromId("test-id"); var prompt = new OrchestrationPrompt("Your message here."); var response = client.executeRequestFromReference(prompt, reference);

warn would be better, debug is disabled most of the time.
Example: we use <root level="INFO"/> for our dependencies.

Jonas-Isr · 2025-12-23T08:29:19Z

orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationClient.java

+  public OrchestrationChatResponse executeRequestFromReference(
+      @Nullable final OrchestrationPrompt prompt,
+      @Nonnull final OrchestrationConfigReference reference) {


We could add

public OrchestrationChatResponse executeRequestFromReference( @Nonnull final OrchestrationConfigReference reference) {...}

as well for even more convenience? (The prompt is used to forward placeholder values and the message history, so it is not always needed but probably in most actual use cases).

I'm not a fan of either method option. Also not liking existing executeRequest siblings.

To me this method looks similar to "normal" use-case

public OrchestrationChatResponse chatCompletion( @Nonnull final OrchestrationPrompt prompt, @Nonnull final OrchestrationModuleConfig config)

As a user i would assume methods have similar signature whether they use an explicit orchestration config, or an implicit config.

Or is the semantic usage different (enough) to justify a different name schema?

Jonas-Isr · 2025-12-23T08:30:59Z

orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationConfigReference.java

+@Value
+@AllArgsConstructor(access = AccessLevel.PRIVATE)
+@Beta
+public class OrchestrationConfigReference {


I decided against making this a sealed interface plus two permitted classes because I felt that would be a bit of an overkill for such little logic. Would be interested in your thoughts about that.

newtork · 2025-12-23T12:08:06Z

.../prompt-registry/src/main/java/com/sap/ai/sdk/prompt/registry/OrchestrationConfigClient.java

+ *
+ * @since 1.14.0
+ */
+public class OrchestrationConfigClient extends OrchestrationConfigsApi {


(Major/Discussion)

I don't think we want to extend a custom class from generated code. Did we do that before? Is there no other way to apply mixins? Otherwise this has a code smell; we're introducing hand-written public API code for the sake of a workaround.

We did that in PromptClient and GroundingClient. It's only to add mix-ins.

newtork · 2025-12-23T12:36:22Z

orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationConfigReference.java

+   */
+  @Nonnull
+  public static Builder fromScenario(@Nonnull final String scenario) {
+    return new Builder(scenario);


(Minor)

You could've turned this into a currying builder, when the classes were interfaces.

return (name) -> (version) -> new OrchestrationConfigReference(null, scenario, name, version);

newtork · 2025-12-23T12:37:35Z

orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationConfigReference.java

+@Value
+@AllArgsConstructor(access = AccessLevel.PRIVATE)
+@Beta
+public class OrchestrationConfigReference {


(Minor)

We could reduce future maintenance if the getters were package scoped(?)

CharlesDuboisSAP

Looking good 😎

CharlesDuboisSAP · 2026-01-07T09:34:34Z

sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OrchestrationService.java

+  private void ensureOrchestrationConfigExists() {
+    final OrchestrationConfigClient orchConfigClient = new OrchestrationConfigClient();
+    if (!orchConfigExists("test-config-for-OrchestrationTest", orchConfigClient)) {
+      final OrchestrationConfigPostRequest postRequest =
+          OrchestrationConfigPostRequest.create()
+              .name("test-config-for-OrchestrationTest")
+              .version("0.0.1")
+              .scenario("sdk-test-scenario")


Suggested change

private void ensureOrchestrationConfigExists() {

final OrchestrationConfigClient orchConfigClient = new OrchestrationConfigClient();

if (!orchConfigExists("test-config-for-OrchestrationTest", orchConfigClient)) {

final OrchestrationConfigPostRequest postRequest =

OrchestrationConfigPostRequest.create()

.name("test-config-for-OrchestrationTest")

.version("0.0.1")

.scenario("sdk-test-scenario")

private void ensureOrchestrationConfigExists(final String scenario, final String name) {

final OrchestrationConfigClient orchConfigClient = new OrchestrationConfigClient();

if (!orchConfigExists("test-config-for-OrchestrationTest", orchConfigClient)) {

final OrchestrationConfigPostRequest postRequest =

OrchestrationConfigPostRequest.create()

.name(name)

.version("0.0.1")

.scenario(scenario)

CharlesDuboisSAP · 2026-01-07T09:36:01Z

sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OrchestrationService.java

+    ensureOrchestrationConfigExists();
+    final var testReference =
+        OrchestrationConfigReference.fromScenario("sdk-test-scenario")
+            .name("test-config-for-OrchestrationTest")
+            .version("0.0.1");
+    final List<Message> history = List.of(new SystemMessage("Start every sentence with an emoji."));
+    final OrchestrationPrompt testPrompt =
+        new OrchestrationPrompt(Map.of("phrase", "Hello World")).messageHistory(history);
+    return client.executeRequestFromReference(testPrompt, testReference);


Suggested change

ensureOrchestrationConfigExists();

final var testReference =

OrchestrationConfigReference.fromScenario("sdk-test-scenario")

.name("test-config-for-OrchestrationTest")

.version("0.0.1");

final List<Message> history = List.of(new SystemMessage("Start every sentence with an emoji."));

final OrchestrationPrompt testPrompt =

new OrchestrationPrompt(Map.of("phrase", "Hello World")).messageHistory(history);

return client.executeRequestFromReference(testPrompt, testReference);

// Could also be enums that are reused in PromptRegistryController

val scenario = "sdk-test-paraphrase";

val name = "create-3-paraphrases-of-sentence";

ensureOrchestrationConfigExists(scenario, name);

final var configReference =

OrchestrationConfigReference.fromScenario(scenario)

.name(name)

.version("0.0.1");

final List<Message> history = List.of(new SystemMessage("Start every sentence with an emoji."));

final OrchestrationPrompt prompt =

new OrchestrationPrompt(Map.of("phrase", "Hello World")).messageHistory(history);

return client.executeRequestFromReference(prompt, configReference);

The current names make it hard to understand what is happening.

CharlesDuboisSAP · 2026-01-07T09:48:21Z

docs/release_notes.md


- [Orchestration] Added new models for `OrchestrationAiModel`: `SAP_ABAP_1`, `SONAR`,`SONAR_PRO`, `GEMINI_2_5_FLASH_LITE`, `CLAUDE_4_5_HAIKU`, `GPT_REALTIME`.
+- [Orchestration] Added new models for `OrchestrationAiModel`: `SAP_ABAP_1`, `SONAR`,`SONAR_PRO`, `GEMINI_2_5_FLASH_LITE`, `CLAUDE_4_5_HAIKU`, `GPT_REALTIME`, `COHERE_COMMAND_A_REASONING`, `NOVA_PREMIER`, `COHERE_RERANKER`.
+- [Orchestration] Configs stored in prompt registry can now be used for Orchestration calls via reference.


Suggested change

- [Orchestration] Configs stored in prompt registry can now be used for Orchestration calls via reference.

- [Orchestration] Configs stored in prompt registry can now be used for Orchestration calls via reference `OrchestrationClient.executeRequestFromReference()`.

Either the added function or a link to the docs would be nice.

CharlesDuboisSAP · 2026-01-07T09:49:16Z

docs/release_notes.md

 - [Orchestration] Convenience for adding the `metadata_params` option to grounding calls.
- [Orchestration] Added new models for `OrchestrationAiModel`: `COHERE_COMMAND_A_REASONING`, `NOVA_PREMIER`, `COHERE_RERANKER`.
 - [Orchestration] Deprecated `DEEPSEEK_R1` model from `OrchestrationAiModel` with no replacement.
+- [Prompt Registry] Added support to manage Orchestration configs stored in Prompt Registry.


Suggested change

- [Prompt Registry] Added support to manage Orchestration configs stored in Prompt Registry.

- [Prompt Registry] Added support to manage Orchestration configs stored in Prompt Registry `OrchestrationConfigClient`.

Either the added class or a link to the docs would be nice.

CharlesDuboisSAP · 2026-01-07T09:54:20Z

orchestration/src/main/java/com/sap/ai/sdk/orchestration/ConfigToRequestTransformer.java

+      if (!prompt.getMessages().isEmpty()) {
+        log.debug(
+            "Messages in prompts are ignored when using Orchestration configs via reference. Change the Orchestration config instead.");
+      }


Can you give a code sample that triggers this warning?

CharlesDuboisSAP · 2026-01-07T09:57:27Z

sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OrchestrationService.java

+                                                    "Create {{?number}} paraphrases of {{?phrase}}"))
+                                            .role(UserChatMessage.RoleEnum.USER))
+                                    .defaults(Map.of("number", "3")))
+                            .model(LLMModelDetails.create().name("gpt-4.1-nano")))));


Suggested change

.model(LLMModelDetails.create().name("gpt-4.1-nano")))));

.model(LLMModelDetails.create().name(GPT_41_NANO.getName())))));

Jonas-Isr added 8 commits December 4, 2025 16:28

add OrchestrationConfigClient, test list, add, delete

3e34453

intermediate

8abe277

clean up code

7039c85

add e2e tests

435aac5

add unit tests

7a6d117

small changes, more tests, and codestyle etc.

e6a8239

Merge branch 'main' into orchestrationConfig-persistence

33ddc76

# Conflicts: # docs/release_notes.md

fix after merge

54849bd

Jonas-Isr self-assigned this Dec 23, 2025

Jonas-Isr added the please-review Request to review a pull-request label Dec 23, 2025

Jonas-Isr commented Dec 23, 2025

View reviewed changes

newtork reviewed Dec 23, 2025

View reviewed changes

CharlesDuboisSAP reviewed Jan 7, 2026

View reviewed changes

	- [Orchestration] Configs stored in prompt registry can now be used for Orchestration calls via reference.
	- [Orchestration] Configs stored in prompt registry can now be used for Orchestration calls via reference `OrchestrationClient.executeRequestFromReference()`.

	- [Prompt Registry] Added support to manage Orchestration configs stored in Prompt Registry.
	- [Prompt Registry] Added support to manage Orchestration configs stored in Prompt Registry `OrchestrationConfigClient`.

	.model(LLMModelDetails.create().name("gpt-4.1-nano")))));
	.model(LLMModelDetails.create().name(GPT_41_NANO.getName())))));

feat: [Orchestration] Support for Orchestration config persistence #697

Are you sure you want to change the base?

feat: [Orchestration] Support for Orchestration config persistence #697

Uh oh!

Conversation

Jonas-Isr commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Feature scope:

Definition of Done

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jonas-Isr Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CharlesDuboisSAP left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Jonas-Isr commented Dec 23, 2025 •

edited

Loading

Jonas-Isr Dec 23, 2025 •

edited

Loading