Skip to content

PR#8 Add support for experiment_types in Bulk API #2002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
khansaad wants to merge 5 commits into
base: runtimes-iirj
Choose a base branch
from
Open

PR#8 Add support for experiment_types in Bulk API #2002

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5292291
feat: add experiment_types support to Bulk API
khansaad Aug 3, 2026
654ea4a
update docs, add string constant, handles null for exp-types
khansaad Aug 10, 2026
f5cbe89
add empty check
khansaad Aug 20, 2026
f57ba94
replace list with enum, add commons library and other fixes
khansaad Aug 27, 2026
02ee149
address review comments
khansaad Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions design/BulkAPI.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,7 @@ progress of the job.
"metadata_profile": "cluster-metadata-local-monitoring",
"measurement_duration": "15min",
"experiment_types": [
"container",
"namespace"
"container"
],
"cluster_name": "prod-cluster",
"model_settings": {
Expand Down Expand Up @@ -94,7 +93,9 @@ progress of the job.

- **datasource:** The data source, e.g., `"Cbank1Xyz"`.

- **experiment_types:** Specifies the type(s) of experiments to run, e.g., `"container"` or `"namespace"`.
- **experiment_types:** Specifies the type of experiment to create for this bulk job. Currently, only a single value is
supported per request — either `"container"` (default) or `"namespace"`. Support for specifying multiple experiment
types in one request will be added in a future release.

- **webhook:** The `webhook` parameter allows the system to notify an external service or consumer about the completion
status of
Expand Down
28 changes: 24 additions & 4 deletions src/main/java/com/autotune/analyzer/serviceObjects/BulkInput.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

import com.autotune.analyzer.kruizeObject.ModelSettings;
import com.autotune.analyzer.kruizeObject.TermSettings;
import com.autotune.analyzer.utils.AnalyzerConstants;
import com.fasterxml.jackson.annotation.JsonIgnore;
import java.util.List;
import java.util.Map;
Expand All @@ -32,26 +33,33 @@ public class BulkInput {
private String metadata_profile;
private String measurement_duration;
private String requestId; //TODO: to be used for the Kafka consumer case to map requestID with jobID

/**
* Cluster name to use for all experiments in this bulk job.
* If provided, overrides cluster name from datasource metadata.
* If not provided, cluster name from metadata will be used.
*/
private String cluster_name;

/**
* Optional model settings to customize which recommendation models to generate.
* If not provided, all models will be generated.
*/
private ModelSettings model_settings;

/**
* Optional term settings to customize which recommendation terms to generate.
* If not provided, all terms will be generated.
*/
private TermSettings term_settings;

/**
* Experiment types to create in this bulk job (e.g. CONTAINER, NAMESPACE).
* If provided, only experiments of the specified type(s) will be created.
* If not provided or empty, defaults to container experiments.
*/
private List<AnalyzerConstants.ExperimentType> experiment_types;

// Getters and Setters

public String getRequestId() {
Expand All @@ -68,7 +76,8 @@ public BulkInput() {
@JsonIgnore
public boolean isEmpty() {
return (filter == null && time_range == null && measurement_duration == null && metadata_profile == null
&& datasource == null && cluster_name == null && model_settings == null && term_settings == null);
&& datasource == null && cluster_name == null && model_settings == null && term_settings == null
&& experiment_types == null);
}

public TimeRange getTime_range() {
Expand Down Expand Up @@ -135,6 +144,17 @@ public void setTerm_settings(TermSettings term_settings) {
this.term_settings = term_settings;
}


public List<AnalyzerConstants.ExperimentType> getExperiment_types() {
return experiment_types;
}

public void setExperiment_types(List<AnalyzerConstants.ExperimentType> experiment_types) {
if (experiment_types != null && !experiment_types.isEmpty()) {
this.experiment_types = experiment_types;
}
}

// Nested class for FilterWrapper that contains 'exclude' and 'include'
public static class FilterWrapper {
private Filter exclude;
Expand Down
13 changes: 12 additions & 1 deletion src/main/java/com/autotune/analyzer/utils/AnalyzerConstants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

import com.autotune.utils.KruizeConstants;
import com.autotune.utils.Utils;
import com.fasterxml.jackson.annotation.JsonCreator;

import java.util.Map;
import java.util.*;
Expand Down Expand Up @@ -326,7 +327,17 @@ public enum ExperimentType {
CONTAINER, // For container-level experiments
NAMESPACE, // For namespace-level experiments
CLUSTER, // For cluster-wide experiments
WORKLOAD // For application-specific experiments
WORKLOAD; // For application-specific experiments

@JsonCreator
public static ExperimentType fromString(String value) {
if (value == null || value.trim().isEmpty()) return null;
try {
return ExperimentType.valueOf(value.trim().toUpperCase());
} catch (IllegalArgumentException e) {
return null;
}
}
Comment on lines +332 to +340

@sourcery-ai sourcery-ai Bot Aug 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): ExperimentType.fromString throws IllegalArgumentException for an invalid or empty experiment_types value, so ObjectMapper.readValue fails before BulkServiceValidation.validateExperimentTypes runs and the API cannot return the intended BULK_INVALID_EXPERIMENT_TYPES validation response.

Triggers: When a request contains an unsupported, empty, or otherwise malformed experiment type.

Suggested fix: Return a nullable/invalid marker that validation can inspect, or catch the enum conversion exception in the request layer and map it to the bulk validation error.

}

/**
Expand Down
1 change: 1 addition & 0 deletions src/main/java/com/autotune/analyzer/utils/AnalyzerErrorConstants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,7 @@ public static final class CreateExperimentAPI {
public static final String MISSING_NAMESPACE_DATA = "Missing NamespaceData for experimentType: %s";
public static final String MISSING_NAMESPACE = "Missing namespace for experimentType: %s";
public static final String INVALID_EXPERIMENT_TYPE = "Invalid experiment_type : %s";
public static final String BULK_INVALID_EXPERIMENT_TYPES = "Invalid experiment_types value(s): %s. Valid values are: container, namespace";

private CreateExperimentAPI() {

Expand Down
142 changes: 124 additions & 18 deletions src/main/java/com/autotune/analyzer/workerimpl/BulkJobManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ public void run() {
setFinalJobStatus(COMPLETED, String.valueOf(HttpURLConnection.HTTP_OK), NOTHING_INFO, datasource);
} else {
jobData.setMetadata(metadataInfo);
Map<String, CreateExperimentAPIObject> createExperimentAPIObjectMap = getExperimentMap(labelString, jobData, metadataInfo, datasource); //Todo Store this map in buffer and use it if BulkAPI pods restarts and support experiment_type
Map<String, CreateExperimentAPIObject> createExperimentAPIObjectMap = getExperimentMap(labelString, jobData, metadataInfo, datasource); //Todo Store this map in buffer and use it if BulkAPI pods restarts
// TODO: Remove getExperimentMap and instead collect all metadata, process it, and create experiments dynamically during metadata iteration.
jobData.getSummary().setTotal_experiments(createExperimentAPIObjectMap.size());
jobData.getSummary().setProcessed_experiments(0);
Expand Down Expand Up @@ -511,6 +511,11 @@ Map<String, CreateExperimentAPIObject> getExperimentMap(String labelString, Bulk
String statusValue = "failure";
Timer.Sample timerGetExpMap = Timer.start(MetricsConfig.meterRegistry());
try {
List<AnalyzerConstants.ExperimentType> experimentTypes = this.bulkInput.getExperiment_types();
AnalyzerConstants.ExperimentType experimentType = (experimentTypes == null || experimentTypes.isEmpty())
? AnalyzerConstants.ExperimentType.CONTAINER
: experimentTypes.get(0);

Map<String, CreateExperimentAPIObject> createExperimentAPIObjectMap = new HashMap<>();
Collection<DataSource> dataSourceCollection = metadataInfo.getDatasources().values();
for (DataSource ds : dataSourceCollection) {
Expand All @@ -521,19 +526,29 @@ Map<String, CreateExperimentAPIObject> getExperimentMap(String labelString, Bulk
: dsc.getDataSourceClusterName();
HashMap<String, DataSourceNamespace> namespaceHashMap = dsc.getNamespaces();
for (DataSourceNamespace namespace : namespaceHashMap.values()) {
HashMap<String, DataSourceWorkload> dataSourceWorkloadHashMap = namespace.getWorkloads();
if (dataSourceWorkloadHashMap != null) {
for (DataSourceWorkload dsw : dataSourceWorkloadHashMap.values()) {
HashMap<String, DataSourceContainer> dataSourceContainerHashMap = dsw.getContainers();
if (dataSourceContainerHashMap != null) {
for (DataSourceContainer dc : dataSourceContainerHashMap.values()) {
// Experiment name - dynamically constructed
String experiment_name = frameExperimentName(labelString, clusterName, namespace, dsw, dc);
// create JSON to be passed in the createExperimentAPI
List<CreateExperimentAPIObject> createExperimentAPIObjectList = new ArrayList<>();
CreateExperimentAPIObject apiObject = prepareCreateExperimentJSONInput(dc, clusterName, dsw, namespace,
experiment_name, createExperimentAPIObjectList);
createExperimentAPIObjectMap.put(experiment_name, apiObject);
if (experimentType == AnalyzerConstants.ExperimentType.NAMESPACE) {
// One namespace experiment per namespace
String experiment_name = frameNamespaceExperimentName(labelString, clusterName, namespace);
List<CreateExperimentAPIObject> createExperimentAPIObjectList = new ArrayList<>();
CreateExperimentAPIObject apiObject = prepareNamespaceExperimentJSONInput(clusterName, namespace,
experiment_name, createExperimentAPIObjectList);
createExperimentAPIObjectMap.put(experiment_name, apiObject);
} else {
// Default: one container experiment per container
HashMap<String, DataSourceWorkload> dataSourceWorkloadHashMap = namespace.getWorkloads();
if (dataSourceWorkloadHashMap != null) {
for (DataSourceWorkload dsw : dataSourceWorkloadHashMap.values()) {
HashMap<String, DataSourceContainer> dataSourceContainerHashMap = dsw.getContainers();
if (dataSourceContainerHashMap != null) {
for (DataSourceContainer dc : dataSourceContainerHashMap.values()) {
// Experiment name - dynamically constructed
String experiment_name = frameExperimentName(labelString, clusterName, namespace, dsw, dc);
// create JSON to be passed in the createExperimentAPI
List<CreateExperimentAPIObject> createExperimentAPIObjectList = new ArrayList<>();
CreateExperimentAPIObject apiObject = prepareCreateExperimentJSONInput(dc, clusterName, dsw, namespace,
experiment_name, createExperimentAPIObjectList);
createExperimentAPIObjectMap.put(experiment_name, apiObject);
}
}
}
}
Expand Down Expand Up @@ -725,19 +740,19 @@ private CreateExperimentAPIObject prepareCreateExperimentJSONInput(DataSourceCon
kubernetesAPIObject.setNamespace(namespace.getNamespace());
kubernetesAPIObjectList.add(kubernetesAPIObject);
createExperimentAPIObject.setKubernetesObjects(kubernetesAPIObjectList);

// Create recommendation settings with threshold
RecommendationSettings rs = new RecommendationSettings();
rs.setThreshold(CREATE_EXPERIMENT_CONFIG_BEAN.getThreshold());

// Pass through model_settings and term_settings from bulk payload if provided
if (bulkInput.getModel_settings() != null) {
rs.setModelSettings(bulkInput.getModel_settings());
}
if (bulkInput.getTerm_settings() != null) {
rs.setTermSettings(bulkInput.getTerm_settings());
}

createExperimentAPIObject.setRecommendationSettings(rs);
TrialSettings trialSettings = new TrialSettings();
trialSettings.setMeasurement_durationMinutes(CREATE_EXPERIMENT_CONFIG_BEAN.getMeasurementDurationStr());
Expand All @@ -752,9 +767,100 @@ private CreateExperimentAPIObject prepareCreateExperimentJSONInput(DataSourceCon
return createExperimentAPIObject;
}


/**
* Builds a CreateExperimentAPIObject for a namespace-level experiment.
* The kubernetes_objects entry contains only a namespaces block (no
* workload name/type or containers), matching the payload expected by
* CreateExperiment for experiment_type "namespace".
*
* @param clusterName resolved cluster name (user override or data source cluster name)
* @param namespace DataSourceNamespace whose namespace is being tracked
* @param experiment_name pre-framed experiment name
* @param createExperimentAPIObjects accumulator list
* @return the constructed CreateExperimentAPIObject
*/
private CreateExperimentAPIObject prepareNamespaceExperimentJSONInput(String clusterName, DataSourceNamespace namespace,
String experiment_name, List<CreateExperimentAPIObject> createExperimentAPIObjects) throws IOException {
CreateExperimentAPIObject createExperimentAPIObject = new CreateExperimentAPIObject();
createExperimentAPIObject.setMode(CREATE_EXPERIMENT_CONFIG_BEAN.getMode());
createExperimentAPIObject.setTargetCluster(CREATE_EXPERIMENT_CONFIG_BEAN.getTarget());
createExperimentAPIObject.setApiVersion(CREATE_EXPERIMENT_CONFIG_BEAN.getVersion());
createExperimentAPIObject.setExperimentName(experiment_name);
createExperimentAPIObject.setDatasource(this.bulkInput.getDatasource());
createExperimentAPIObject.setClusterName(clusterName);
createExperimentAPIObject.setPerformanceProfile(CREATE_EXPERIMENT_CONFIG_BEAN.getPerformanceProfile());
createExperimentAPIObject.setMetadataProfile(CREATE_EXPERIMENT_CONFIG_BEAN.getMetadataProfile());

// Namespace experiment: kubernetes_objects has only a namespaces block, no containers
List<KubernetesAPIObject> kubernetesAPIObjectList = new ArrayList<>();
KubernetesAPIObject kubernetesAPIObject = new KubernetesAPIObject();
NamespaceAPIObject namespaceAPIObject = new NamespaceAPIObject(namespace.getNamespace(), null, null);
kubernetesAPIObject.setNamespaceAPIObject(namespaceAPIObject);
kubernetesAPIObjectList.add(kubernetesAPIObject);
createExperimentAPIObject.setKubernetesObjects(kubernetesAPIObjectList);

// Recommendation settings
RecommendationSettings rs = new RecommendationSettings();
rs.setThreshold(CREATE_EXPERIMENT_CONFIG_BEAN.getThreshold());
if (this.bulkInput.getModel_settings() != null) {
rs.setModelSettings(this.bulkInput.getModel_settings());
}
if (this.bulkInput.getTerm_settings() != null) {
rs.setTermSettings(this.bulkInput.getTerm_settings());
}
createExperimentAPIObject.setRecommendationSettings(rs);

TrialSettings trialSettings = new TrialSettings();
trialSettings.setMeasurement_durationMinutes(CREATE_EXPERIMENT_CONFIG_BEAN.getMeasurementDurationStr());
createExperimentAPIObject.setTrialSettings(trialSettings);

createExperimentAPIObject.setExperiment_id(Utils.generateID(createExperimentAPIObject.toString()));
createExperimentAPIObject.setStatus(AnalyzerConstants.ExperimentStatus.IN_PROGRESS);
createExperimentAPIObject.setExperimentType(AnalyzerConstants.ExperimentType.NAMESPACE);

createExperimentAPIObjects.add(createExperimentAPIObject);
return createExperimentAPIObject;
}

/**
* Frames the experiment name for a namespace-level experiment.
* Uses datasource, cluster name, and namespace — workload/container
* segments are not meaningful for namespace experiments.
*
* @param labelString label filter string (may be null)
* @param clusterName resolved cluster name (user override or data source cluster name)
* @param namespace namespace metadata
* @return framed experiment name
*/
public String frameNamespaceExperimentName(String labelString, String clusterName,
DataSourceNamespace namespace) {
String datasource = this.bulkInput.getDatasource();
String namespaceName = namespace.getNamespace();

// Namespace experiment name: datasource|clustername|namespace
String experimentName = KruizeDeploymentInfo.namespace_experiment_name_format
.replace("%datasource%", datasource)
.replace("%clustername%", clusterName)
.replace("%namespace%", namespaceName);
Comment on lines +841 to +845

@sourcery-ai sourcery-ai Bot Aug 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (broader_impact): Namespace experiments use the datasource metadata cluster in their experiment name, while prepareNamespaceExperimentJSONInput uses the trimmed bulkInput.cluster_name in the experiment payload. With a cluster override, the created experiment name identifies one cluster but its cluster_name field identifies another, breaking the name/cluster identity invariant and causing inconsistent lookups or duplicate experiments.

Triggers: When a namespace bulk request supplies cluster_name that differs from the cluster name in datasource metadata.

Suggested fix: Resolve and trim the cluster name once, pass it to frameNamespaceExperimentName, and use that same value for both the experiment name and payload.


if (null != labelString) {
Map<String, String> labelsMap = parseLabelString(labelString);
Pattern labelPattern = Pattern.compile("%label:([a-zA-Z0-9_]+)%");
Matcher matcher = labelPattern.matcher(experimentName);
while (matcher.find()) {
String labelKey = matcher.group(1);
String labelValue = labelsMap.getOrDefault(labelKey, "unknown" + labelKey);
experimentName = experimentName.replace(matcher.group(), labelValue != null ? labelValue : "unknown" + labelKey);
}
}
LOGGER.debug("Namespace experiment name: {}", experimentName);
return experimentName;
}

/**
* @param labelString
* @param dataSourceCluster
* @param clusterName
* @param dataSourceNamespace
* @param dataSourceWorkload
* @param dataSourceContainer
Expand Down
Loading
Loading