Skip to content

[LIVY-1061] Add livy.server.app-name-prefix for interactive session YARN app names - #534

Merged
gyogal merged 3 commits into
apache:masterfrom
samuhasavak5:master-1061
Jul 28, 2026
Merged

[LIVY-1061] Add livy.server.app-name-prefix for interactive session YARN app names#534
gyogal merged 3 commits into
apache:masterfrom
samuhasavak5:master-1061

Conversation

@samuhasavak5

@samuhasavak5 samuhasavak5 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This PR adds a new Livy server configuration property, livy.server.app-name-prefix, to allow cluster administrators to customize the default YARN application name for interactive Spark sessions.

Problem: When a client creates an interactive session via POST /sessions without specifying a name, Livy sets spark.app.name to the hardcoded value livy-session-. In HA or multi-cluster environments, this makes it difficult to distinguish Livy-managed applications in the YARN Resource Manager UI.

Solution:

Add LivyConf.SERVER_APP_NAME_PREFIX (livy.server.app-name-prefix, default: livy-session)
Use the configured prefix when building the default spark.app.name in InteractiveSession.create()
Document the new property in conf/livy.conf.template
Add unit tests for default and custom prefix behavior
Scope: Interactive sessions only. Batch sessions already support per-request naming via the REST API name field and are unchanged.

Backward compatibility: When the property is unset, behavior remains livy-session-.

Example:
livy.conf

livy.server.app-name-prefix = my-cluster

Creating a session without a name yields YARN application name: my-cluster-3

JIRA: LIVY-1061

How was this patch tested?

  • Unit tests: Added tests in InteractiveSessionSpec for:
    -Default prefix (livy-session) when config is not set
    -Custom prefix (e.g. my-cluster) when livy.server.app-name-prefix is configured
  • Build: mvn package -Pspark3 -Pscala-2.12 — livy-server and dependent modules built successfully
  • Manual verification (recommended): Set livy.server.app-name-prefix in livy.conf, create an interactive session without name, and confirm the YARN RM UI shows -

No UI changes in this PR.

Was this patch authored or co-authored using generative AI tooling?

Yes, this was co-authored using Cursor to help generate the new test cases.

@samuhasavak5 samuhasavak5 changed the title CDPD-103411-Fixed Livy app name [LIVY-1061] Add livy.server.app-name-prefix for interactive session YARN app names Jul 27, 2026
@samuhasavak5

Copy link
Copy Markdown
Contributor Author

@roczei , @gyogal , @nileshrathi345 and @ArnavBalyan
could you please take a look when possible and review
thanks..!!!!

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.63%. Comparing base (0efa2d7) to head (07db93b).
⚠️ Report is 30 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master     #534      +/-   ##
============================================
+ Coverage     68.38%   68.63%   +0.25%     
- Complexity     1199     1215      +16     
============================================
  Files           106      106              
  Lines          6711     6785      +74     
  Branches        831      834       +3     
============================================
+ Hits           4589     4657      +68     
- Misses         1657     1660       +3     
- Partials        465      468       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@roczei roczei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samuhasavak5

Thanks for this! Nice, focused change with good tests and docs. LGTM

@gyogal gyogal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for your contribution @samuhasavak5 !

@gyogal
gyogal merged commit 9e18a86 into apache:master Jul 28, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants