Skip to content

Comments

sqlite: optimize column name creation and text value encoding#61954

Open
thisalihassan wants to merge 1 commit intonodejs:mainfrom
thisalihassan:sqlite-one-byte-encoding
Open

sqlite: optimize column name creation and text value encoding#61954
thisalihassan wants to merge 1 commit intonodejs:mainfrom
thisalihassan:sqlite-one-byte-encoding

Conversation

@thisalihassan
Copy link
Contributor

@thisalihassan thisalihassan commented Feb 23, 2026

Skip the full UTF-8 decode path for text values that are pure ASCII by validating with simdutf and creating OneByte V8 strings, halving memory.Internalize column name strings so V8 shares hidden classes across row objects. Cache column names in the iterate() hot loop, invalidating on schema changes via SQLITE_STMTSTATUS_REPREPARE.

Benchmark: 30-run
sqlite-benchmark

Refs: nodejs/performance#181

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/sqlite

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. sqlite Issues and PRs related to the SQLite subsystem. labels Feb 23, 2026
@thisalihassan thisalihassan force-pushed the sqlite-one-byte-encoding branch from 6adb625 to ff6a788 Compare February 23, 2026 14:58
Use simdutf to detect ASCII text values and create them
via NewFromOneByte for compact one-byte representation.
Internalize column name strings with kInternalized so V8
shares hidden classes across row objects. Cache column
names on StatementSync for iterate(), invalidated via
SQLITE_STMTSTATUS_REPREPARE on schema changes.

Refs: nodejs/performance#181
@thisalihassan thisalihassan force-pushed the sqlite-one-byte-encoding branch from ff6a788 to 4fcb1ed Compare February 23, 2026 15:05
@thisalihassan
Copy link
Contributor Author

thisalihassan commented Feb 23, 2026

I didn't push the changes for sqlite-prepare-select-all-options and sqlite-prepare-select-all where I tested the iterate method because it slows down the benchmark by a lot.
Adding iterate nearly ~2.3x the total benchmark runtime from ~40 minutes to ~90 minutes, but if needed I can push that as well
Benchmark with iterate: 30 runs
sqlite-benchmark_all_2

@codecov
Copy link

codecov bot commented Feb 23, 2026

Codecov Report

❌ Patch coverage is 82.92683% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.77%. Comparing base (da5efc4) to head (4fcb1ed).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
src/node_sqlite.cc 82.92% 4 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #61954      +/-   ##
==========================================
+ Coverage   88.84%   89.77%   +0.93%     
==========================================
  Files         674      674              
  Lines      204957   205638     +681     
  Branches    39309    39415     +106     
==========================================
+ Hits       182087   184605    +2518     
+ Misses      15088    13273    -1815     
+ Partials     7782     7760      -22     
Files with missing lines Coverage Δ
src/node_sqlite.h 81.48% <ø> (+1.08%) ⬆️
src/node_sqlite.cc 80.91% <82.92%> (+0.01%) ⬆️

... and 179 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ronag ronag requested review from addaleax, anonrig and geeksilva97 and removed request for geeksilva97 February 24, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. sqlite Issues and PRs related to the SQLite subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants