Skip to content

perf: enable OPcache and Hyperloop B #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
anurag6569201 wants to merge 1 commit into
base: qa/agent-appwrite-appwrite/pr-11-13487/base
Choose a base branch
from
Open

perf: enable OPcache and Hyperloop B #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ _APP_NOTIFICATIONS_TRACKING_SECRET=your-secret-key
_APP_RULES_FORMAT=md5
_APP_SYSTEM_RESPONSE_FORMAT=
_APP_TRUSTED_HEADERS=x-forwarded-for
_APP_POOL_ADAPTER=stack
_APP_WORKER_PER_CORE=6
_APP_WORKERS_NUM=1
_APP_WORKER_MAX_COROUTINES=1
Expand Down
4 changes: 4 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ COPY ./public /usr/src/code/public
COPY ./bin /usr/local/bin
COPY ./src /usr/src/code/src
COPY ./dev /usr/src/code/dev
COPY ./docker/opcache.ini /usr/local/etc/php/conf.d/zz-opcache.ini
COPY ./mongo-init.js /usr/src/code/mongo-init.js
COPY ./mongo-entrypoint.sh /usr/src/code/mongo-entrypoint.sh

Expand Down Expand Up @@ -121,6 +122,9 @@ FROM appwrite/base:2.0.0-xdebug AS xdebug

FROM base AS development

# Revalidate bind-mounted source files when development workers reload.
RUN printf 'opcache.validate_timestamps=1\nopcache.revalidate_freq=0\n' > /usr/local/etc/php/conf.d/zzz-opcache-dev.ini

COPY ./docs /usr/src/code/docs
COPY ./dev /usr/src/code/dev

Expand Down
4 changes: 3 additions & 1 deletion app/http.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
use Utopia\Database\Helpers\Permission;
use Utopia\Database\Helpers\Role;
use Utopia\DI\Container;
use Utopia\Http\Adapter\Swoole\Mode;
use Utopia\Http\Adapter\Swoole\Server;
use Utopia\Http\Files;
use Utopia\Http\Http;
Expand Down Expand Up @@ -54,8 +55,9 @@
host: "0.0.0.0",
port: System::getEnv('PORT', 80),
settings: [
// Apply Cloud's coroutine preset, retaining Appwrite's worker and payload limits.
...Mode::HYPERLOOP_B->settings(),
Constant::OPTION_WORKER_NUM => $totalWorkers,
Constant::OPTION_HTTP_COMPRESSION => false,
Constant::OPTION_PACKAGE_MAX_LENGTH => $payloadSize,
Constant::OPTION_OUTPUT_BUFFER_SIZE => $payloadSize,
],
Expand Down
5 changes: 1 addition & 4 deletions app/init/registers.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
use Utopia\Http\Http;
use Utopia\Messaging\Adapter\Email\SMTP;
use Utopia\Mongo\Client as MongoClient;
use Utopia\Pools\Adapter\Stack as StackPool;
use Utopia\Pools\Adapter\Swoole as SwoolePool;
use Utopia\Pools\Group;
use Utopia\Pools\Pool;
Expand Down Expand Up @@ -223,16 +222,14 @@
},
};

$poolAdapter = System::getEnv('_APP_POOL_ADAPTER', default: 'stack') === 'swoole' ? new SwoolePool() : new StackPool();

// PubSub workers hold one long-lived subscribed connection and also need
// spare capacity for publishes from the same process.
$connectionPoolSize = match ($type) {
'pubsub' => max(2, $poolSize),
default => $poolSize,
};

$pool = new Pool($poolAdapter, $name, $connectionPoolSize, function () use ($type, $resource, $dsn) {
$pool = new Pool(new SwoolePool(), $name, $connectionPoolSize, function () use ($type, $resource, $dsn) {
// Get Adapter
switch ($type) {
case 'database':
Expand Down
22 changes: 0 additions & 22 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,6 @@ services:
- _APP_STATS_RESOURCES_INTERVAL
- _APP_EDITION
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_LOCALE
- _APP_COMPRESSION_ENABLED
- _APP_COMPRESSION_MIN_SIZE_BYTES
Expand Down Expand Up @@ -353,7 +352,6 @@ services:
- _APP_DOCUMENTSDB
- _APP_VECTORSDB
- _APP_EMBEDDING
- _APP_POOL_ADAPTER=swoole

appwrite-worker:
entrypoint: worker
Expand Down Expand Up @@ -398,7 +396,6 @@ services:
# Pool sizing only (sum of per-queue caps). Each queue keeps its own maxCoroutines in PHP; databases stays at 1.
- _APP_WORKER_MAX_COROUTINES=78
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_EMAIL_SECURITY
- _APP_EMAIL_CERTIFICATES
Expand Down Expand Up @@ -555,7 +552,6 @@ services:
# against shared console/cache pools; floor must cover concurrent ticks.
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_OPTIONS_FORCE_HTTPS
- _APP_DOMAIN
Expand Down Expand Up @@ -622,7 +618,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_EMAIL_SECURITY
- _APP_DB_ADAPTER
Expand Down Expand Up @@ -674,7 +669,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_REDIS_HOST
- _APP_REDIS_PORT
Expand Down Expand Up @@ -751,7 +745,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_REDIS_HOST
- _APP_REDIS_PORT
Expand Down Expand Up @@ -810,7 +803,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_EXECUTOR_SECRET
- _APP_EXECUTOR_HOST
Expand Down Expand Up @@ -911,7 +903,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_STORAGE_DEVICE
- _APP_STORAGE_S3_ACCESS_KEY
Expand Down Expand Up @@ -980,7 +971,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_LOGGING_CONFIG
- _APP_LOGGING_FORMAT
- _APP_OPENSSL_KEY_V1
Expand Down Expand Up @@ -1043,7 +1033,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_DOMAIN
- _APP_DOMAIN_TARGET_CNAME
Expand Down Expand Up @@ -1095,7 +1084,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_REDIS_HOST
- _APP_REDIS_PORT
Expand Down Expand Up @@ -1132,7 +1120,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=8
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_DOMAIN
- _APP_OPTIONS_FORCE_HTTPS
Expand Down Expand Up @@ -1179,7 +1166,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=6
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_NOTIFICATIONS_TRACKING_SECRET
- _APP_SYSTEM_EMAIL_NAME
Expand Down Expand Up @@ -1225,7 +1211,6 @@ services:
environment:
- _APP_ENV
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_NOTIFICATIONS_TRACKING_SECRET
- _APP_SYSTEM_EMAIL_NAME
Expand Down Expand Up @@ -1273,7 +1258,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_REDIS_HOST
- _APP_REDIS_PORT
Expand Down Expand Up @@ -1337,7 +1321,6 @@ services:
- _APP_WORKERS_NUM=1
- _APP_WORKER_MAX_COROUTINES=1
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_DOMAIN
- _APP_DOMAIN_TARGET_CNAME
Expand Down Expand Up @@ -1396,7 +1379,6 @@ services:
- _APP_ENV
- _APP_LOGGING_FORMAT
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_DOMAIN
- _APP_DOMAIN_TARGET_CNAME
- _APP_DOMAIN_TARGET_AAAA
Expand Down Expand Up @@ -1446,7 +1428,6 @@ services:
- _APP_ENV
- _APP_LOGGING_FORMAT
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_DOMAIN
- _APP_DOMAIN_TARGET_CNAME
- _APP_DOMAIN_TARGET_AAAA
Expand Down Expand Up @@ -1491,7 +1472,6 @@ services:
- _APP_ENV
- _APP_LOGGING_FORMAT
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_OPTIONS_FORCE_HTTPS
- _APP_DOMAIN
Expand Down Expand Up @@ -1533,7 +1513,6 @@ services:
- _APP_ENV
- _APP_LOGGING_FORMAT
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_OPTIONS_FORCE_HTTPS
- _APP_DOMAIN
Expand Down Expand Up @@ -1574,7 +1553,6 @@ services:
- _APP_ENV
- _APP_LOGGING_FORMAT
- _APP_WORKER_PER_CORE
- _APP_POOL_ADAPTER
- _APP_OPENSSL_KEY_V1
- _APP_REDIS_HOST
- _APP_REDIS_PORT
Expand Down
9 changes: 9 additions & 0 deletions docker/opcache.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
opcache.enable=1
opcache.enable_cli=1
opcache.memory_consumption=384
opcache.interned_strings_buffer=32
opcache.max_accelerated_files=65536
opcache.validate_timestamps=0
opcache.save_comments=1
opcache.jit=tracing
opcache.jit_buffer_size=128M
14 changes: 9 additions & 5 deletions src/Appwrite/Platform/Modules/Storage/Http/Buckets/Files/Create.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,15 @@ public function action(
}
}

$chunksUploaded = max($uploaded, $chunksUploaded, (int) ($metadata['chunks'] ?? 0));
// Another chunk may have finalized the upload and removed its parts

@shipwright-agent shipwright-agent Bot Sep 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipwright · CRITICAL

The new guard 'if (empty($chunksUploaded))' is placed before '$chunksUploaded' is assigned in the chunked-upload path.

Impact: The new guard 'if (empty($chunksUploaded))' is placed before '$chunksUploaded' is assigned in the chunked-upload path. In the original code, '$chunksUploaded' was initialized from 'max($uploaded, $chunksUploaded, (int) ($metadata['chunks'] ?? 0))' before any check. The diff removes that initialization and instead checks 'empty($chunksUploaded)' first, so on the first chunk '$chunksUploaded' is still null/0 and the r…

Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.

@shipwright-agent shipwright-agent Bot Sep 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shipwright · CRITICAL

The new guard 'if (empty($chunksUploaded))' is placed before '$chunksUploaded' is assigned in the chunked-upload path.

Impact: The new guard 'if (empty($chunksUploaded))' is placed before '$chunksUploaded' is assigned in the chunked-upload path. In the original code, '$chunksUploaded' was initialized from 'max($uploaded, $chunksUploaded, (int) ($metadata['chunks'] ?? 0))' before any check. The diff removes that initialization and instead checks 'empty($chunksUploaded)' first, so on the first chunk '$chunksUploaded' is still null/0 and the r…

Suggested fix: Fix the review finding before release.

// before upload() counted them. Check the completed document above first.
if (empty($chunksUploaded)) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed uploading file');
}

// A local chunk file is visible before its write finishes. Only parts
// recorded after upload() returns are safe to include in finalization.
$chunksUploaded = max($uploaded, isset($metadata['parts']) ? \count($metadata['parts']) : $chunksUploaded);

if ($chunksUploaded === $chunks && $uploaded < $chunks) {
$deviceForFiles->finalize($path, $chunks, $metadata);
Expand Down Expand Up @@ -526,10 +534,6 @@ public function action(
$metadata
);

if (empty($chunksUploaded)) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed uploading file');
}

$locks($lockKey, 600, fn () => $finalizeUpload($chunksUploaded), timeout: 120.0);
} catch (LockContention) {
$response->addHeader('Retry-After', '5');
Expand Down