跳到主要内容

Databend Releases

This page provides information about recent features, enhancements, and bug fixes for Databend.

May 6, 2024 (v1.2.452)

What's Changed

Exciting New Features ✨

  • feat(query): implement ST_GEOMPOINTFROMGEOHASH by @kkk25641463 in #15182
  • feat(rbac): drop role will transfer role owns object to account_admin by @TCeason in #15154
  • feat(query): implement ST_MAKEPOLYGON by @kkk25641463 in #15194
  • feat: distinguish early close from kill/cancel with new LogType::Closed. by @youngsofun in #15206
  • feat: add EXECUTE IMMEDIATE statement by @andylokandy in #15144
  • feat(http): get client ip from headers by @flaneur2020 in #15205
  • feat(query): support alter table comment by @TCeason in #15227
  • feat(query): match function support multiple fields with boost by @b41sh in #15196
  • feat: support insert with cte by @xudong963 in #15241
  • feat: add setting script_max_steps by @andylokandy in #15244
  • feat: atomic CTAS by @dantengsky in #15186
  • feat(query): integrate "like" to filter execution by @Dousir9 in #15233
  • feat(query): add user option: disabled by @TCeason in #15249
  • feat(query): Inverted index support set filters and index record options by @b41sh in #15254
  • feat: add sequence meta api by @lichuang in #15247
  • feat(query): add new setting parse_datetime_ignore_remainder. by @TCeason in #15260
  • feat: http handler response add field has_result_set and state Starting. by @youngsofun in #15258
  • feat: support cte in update selection by @xudong963 in #15276
  • feat: impl create dynamic table sql parser and planner by @zhyass in #15250
  • feat: support cte in delete by @xudong963 in #15284
  • feat(query): implement ST_ASGEOJSON by @kkk25641463 in #15214
  • feat(query): support map_keys and map_values function by @b41sh in #15291
  • feat(query): implement TO_GEOMETRY/TRY_TO_GEOMETRY function by @kkk25641463 in #15302
  • feat(query): 14925, support udf wasm by @shamb0 in #15107
  • feat(query): top k syntax support by @TCeason in #15318
  • feat(query): implement st_asewkb/st_aswkb/st_asewkt/st_aswkt/st_geohash by @kkk25641463 in #15325
  • feat(query): support with clause in copy into by @sundy-li in #15343
  • feat(query): inverted index support json type by @b41sh in #15267
  • feat(query): support compute quota feature by @zhang2014 in #15336
  • feat: Introduce enterprise storage quota feature by @Xuanwo in #15338
  • feat(query): inverted index use empty position data when query not contain phrase terms by @b41sh in #15362
  • feat(query): implement ST_DIMENSION function by @kkk25641463 in #15347
  • feat(query): implement ST_ENDPOINT by @kkk25641463 in #15366
  • feat(query): implement ST_POINTN by @kkk25641463 in #15373
  • feat(query): support explain insert with insert source by @TCeason in #15357
  • feat(query): implement ST_SRID by @kkk25641463 in #15377
  • feat(query): implement ST_X/ST_Y/ST_STARTPOINT by @kkk25641463 in #15384
  • feat: Add: stream based snapshot API install_snapshot_v2() to meta-service by @drmingdrmer in #15395

Thoughtful Bug Fix 🔧

  • fix(query): set setting case insensitive by @TCeason in #15199
  • fix(query): show grants panic when role/user has immute db/table privilege by @TCeason in #15188
  • fix: geos-sys compile error #15200 by @drmingdrmer in #15209
  • fix(query): alter modify col comment need modify when type not modify by @TCeason in #15217
  • fix(query): fix negative session_running_acquired_queries by @zhang2014 in #15223
  • fix: fix map get return double nested nullable by @ariesdevil in #15230
  • fix: show locks in current tenant by @zhyass in #15252
  • fix: solve some inconsistencies between code and comments by @YichiZhang0613 in #15229
  • fix: fix delete from subquery and one more condition bug by @lichuang in #15279
  • fix(query): fix udf name with upper case letters by @b41sh in #15300
  • fix: don't constant fold giant expr by @andylokandy in #15311
  • fix(query): fix decimal op loss precision by @sundy-li in #15313
  • fix(query): NDJSON copy into allow cast bool and number to string by @b41sh in #15308
  • fix: incorrect overlap checking (replace into) by @dantengsky in #15315
  • fix(typo): refering -> referring by @andylokandy in #15323
  • fix: insert Plan should not have schema. by @youngsofun in #15330
  • fix(executor): queries exeuctor points calc incorrectly by @dqhl76 in #15349
  • fix(query): show processlit host is null in HTTPQuery by @TCeason in #15363
  • fix: check all partitions for small files instead of the last only. by @youngsofun in #15360
  • fix: structlog should print non-zero-rooted tree by @andylokandy in #15370
  • fix: when listing-kv, limit the right bound if possible by @drmingdrmer in #15372
  • fix: list stage returns at most 15000 files. by @youngsofun in #15385
  • fix(query): spill block need consider scalar by @TCeason in #15387
  • fix: Respond with SnapshotMismatch error instead of internal error by @drmingdrmer in #15391
  • fix: solve some inconsistencies between code and comments by @YichiZhang0613 in #15394

Code Refactor 🎉

  • refactor(query): remove useless code (system.tracing) by @zhang2014 in #15185
  • refactor: explain merge into by @xudong963 in #15195
  • refactor: simplify DatabaseFactory, just static dispatch by @drmingdrmer in #15203
  • refactor(query): move api and metrics to servers mod by @zhang2014 in #15210
  • refactor: add view system tables by @TCeason in #15184
  • refactor: introduce UninitTenant and TIdentRaw by @drmingdrmer in #15207
  • refactor: define DatabaseNameIdent with TIdent by @drmingdrmer in #15235
  • refactor: ScalarVisitor refactor push_down_prewhere by @TCeason in #15215
  • refactor: replace String with Tenant in xxxPlan by @drmingdrmer in #15240
  • refactor: avoid using PREFIX directly. use method to build list-prefix by @drmingdrmer in #15243
  • refactor: remove unused list_all_tables() by @drmingdrmer in #15251
  • refactor: remove kvapi::Key impl for String by @drmingdrmer in #15253
  • refactor: grant_ownership retry logic by @TCeason in #15275
  • refactor: Extract kvapi::KeyCodec trait from kvapi::Key by @drmingdrmer in #15262
  • refactor: Introduce generic parameter N to TIdent by @drmingdrmer in #15271
  • refactor: define ShareConsumer with TIdent by @drmingdrmer in #15280
  • refactor: define VirtualColumnNameIdent with TIdent by @drmingdrmer in #15285
  • refactor: define DbIdListKey with TIdent, and rename it to DatabaseIdHistoryIdent by @drmingdrmer in #15287
  • refactor: define TenantQuotaIdent with TIdent by @drmingdrmer in #15289
  • refactor: Define TableLockKey with TIdent by @drmingdrmer in #15319
  • refactor: TIdent add a flag to indicate if the tenant should be encoded into key by @drmingdrmer in #15321
  • refactor: define BackgroundJobIdIdent with TIdent, with HAS_TENANT disabled by @drmingdrmer in #15327
  • refactor: http handler check has_result_set() before get schema of plan. by @youngsofun in #15329
  • refactor: Define DataMaskIdIdent with TIdent by @drmingdrmer in #15331
  • refactor: add right bound when listing expired keys to avoid unnecessary data copy by @drmingdrmer in #15334
  • refactor: Upgrade Openraft to 0.9.7 by @drmingdrmer in #15339
  • refactor: improve install-snapshot performance by @drmingdrmer in #15345
  • refactor: move snapshot building io to other task. by @drmingdrmer in #15351
  • refactor: only has password policy update user login res by @TCeason in #15356
  • refactor: Define TenantOwnershiptObject with TIdent by @drmingdrmer in #15358
  • refactor: define TenantUserIdent with TIdent by @drmingdrmer in #15361
  • refactor: session_ctx.client_ip refactor to Option<String> by @TCeason in #15368
  • refactor: move DirName to separate file; add peek() to KeyParser by @drmingdrmer in #15365
  • refactor: Define StageFileIdent with TIdent by @drmingdrmer in #15371
  • refactor: remove stage use stream to avoid OOM. by @youngsofun in #15378
  • refactor: sync create inverted index by @b41sh in #15379
  • refactor: dispatch load to 16 threads when installing snapshot by @drmingdrmer in #15381
  • refactor: get_table_by_id() returns an Option<Seqv<TableMeta>> by @drmingdrmer in #15388
  • refactor: remove get_table_name_by_id() by @drmingdrmer in #15389

Others 📒

  • chore(executor): add time limit daemon for queries executor by @dqhl76 in #15174
  • chore(query): max_running_queries from 0 to 8, enable the max running queries by @zhang2014 in #15190
  • chore(query): add session_running_acquired_queries metric by @zhang2014 in #15193
  • chore(executor): add end_of_stream flag for executor stream by @zhang2014 in #15202
  • chore(planner): refine explain read size by @Dousir9 in #15204
  • chore: change parquet_fast_read_bytes setting from 0 to 16MB by @BohuTANG in #15212
  • chore(planner): fix decorrelate exists subquery by @Dousir9 in #15208
  • chore: old op LogicalGet modify to new op name Scan by @TracyZYJ in #15222
  • chore(query): parser error need to log error by @sundy-li in #15219
  • chore: vacuum drop table for external location table by @BohuTANG in #15220
  • chore(query): fix nullable array domain by @sundy-li in #15224
  • chore(query): add query queued duration for query log by @zhang2014 in #15225
  • chore(query): remove useless function map:get by @sundy-li in #15245
  • chore: update sqllogictests crate to 0.19.1 by @finchxx in #15242
  • chore(query): support like push down not by @Dousir9 in #15268
  • chore(executor): add node level error tracking by @dqhl76 in #15239
  • chore(query): refine set sql_dialect error message by @Dousir9 in #15282
  • chore(query): bump arrow-udf to extra lib by @sundy-li in #15283
  • chore(planner): improve delete explain by @Dousir9 in #15290
  • chore: Enhance Flexibility of Dynamic Table Creation by Allowing Options in Any Order by @zhyass in #15297
  • chore: add license to cargo.toml by @BohuTANG in #15299
  • chore(planner): fix can_filter_null by @Dousir9 in #15298
  • chore: improve distributed plan by @xudong963 in #15296
  • chore(query): simplify stage list by @sundy-li in #15303
  • chore: enable distributed optimization for subquery in delete by @xudong963 in #15288
  • chore: correct stream error message by @zhyass in #15309
  • chore: tune auto-compact parameters triggered by insertions by @dantengsky in #15261
  • chore(query): improve delete pruner by @sundy-li in #15316
  • chore(planner): fix merge into statistics by @Dousir9 in #15320
  • chore(planner): add disable merge into join reorder settings by @Dousir9 in #15322
  • chore: fix RulePushDownLimitWindow id by @xudong963 in #15326
  • chore(query): add merge ratio in sort spill by @sundy-li in #15332
  • chore: refine runtime filter log by @xudong963 in #15335
  • chore: add err message to help capture new agg ht hang by @Freejww in #15344
  • chore: enhance runtime filter log by @xudong963 in #15350
  • chore(planner): support fast return for hash join spill by @Dousir9 in #15364
  • chore(ci): fix flaky test; by @TCeason in #15369
  • chore: add more thest to prefix_right_bound() by @drmingdrmer in #15374
  • chore: purge inverted index by @dantengsky in #15354
  • chore(query): reuse the script runtime during pipeline builder by @sundy-li in #15375
  • chore: adjust row count assertion in table compaction by @dantengsky in #15380
  • chore: if a txn log is too long, display only the first 5 items by @drmingdrmer in #15392
  • chore: Fix opendal upstream commit missing by @Xuanwo in #15393

New Contributors

  • @finchxx made their first contribution in #15242
  • @YichiZhang0613 made their first contribution in #15229
  • @shamb0 made their first contribution in #15107

Full Changelog: https://github.com/datafuselabs/databend/compare/v1.2.410...v1.2.452

Apr 8, 2024 (v1.2.410)

What's Changed

Exciting New Features ✨

  • feat(query): support settings admin api for global level settings by @zhang2014 in #14901
  • feat: add a new ratio config instead of table_data_deserialized_data_bytes by @chienguo in #14896
  • feat(ast): add parser for script by @andylokandy in #14891
  • feat: support cross join spilling by @xudong963 in #14887
  • feat: add more function alias names by @sundy-li in #14924
  • feat(query): support spill for new agg hashtable by @Freejww in #14905
  • feat(query): Support refresh inverted index by @b41sh in #14912
  • feat: ndjson support option null_if. by @youngsofun in #14938
  • feat(query): add statement_queued_timeout_in_seconds setting for queries queue by @zhang2014 in #14945
  • feat: change tracking enabled table support merge into by @zhyass in #14900
  • feat: histogram aggregate function by @suimenno3002 in #14839
  • feat(query): add metrics for session queue manager by @everpcpc in #14966
  • feat(query): add error_or function by @sundy-li in #14980
  • feat: purge duplicated files (copy into table) by @dantengsky in #14962
  • feat(query): support parse mulity modify column? ty by @TracyZYJ in #14979
  • feat: support prql by @ncuwaln in #14922
  • chore(query): support domain contains in string type by @sundy-li in #15023
  • feat(query): check view's inner table privilege when create or alter view by @TCeason in #14986
  • feat: add buffer for spiller by @xudong963 in #15021
  • feat: add show views & desc view by @blackstar-baba in #14926
  • feat: support pagination on task_history by @ZhiHanZ in #15047
  • feat(ast): add parser for template sql by @andylokandy in #15026
  • refactor: SHOW GRANTS should not display drop on table/database by @TCeason in #14931
  • feat(query): Support query match inverted index by @b41sh in #14997
  • feat: Implement concurrent stat by @Xuanwo in #15083
  • feat(metasrv): add network metrics rpc_delay_ms by @everpcpc in #15084
  • feat: create stream at navigation point by @zhyass in #15062
  • feat(query): implement st_makeline() function by @kkk25641463 in #15059
  • feat(planner): improve push down filter join by @Dousir9 in #14872
  • feat: support otel context propagation for databend-query by @ZhiHanZ in #15096
  • feat(query): add query_result_cache_min_execute_secs by @sundy-li in #15101
  • feat(query): Support hook refresh inverted index after insert by @b41sh in #15094
  • feat: time travel support at stream by @zhyass in #15116
  • feat(query): implement st_geometryfromewkb function by @kkk25641463 in #15118
  • feat: refactor ndjson input format. by @youngsofun in #14943
  • feat(script): add script compiler by @andylokandy in #15113
  • feat: support multi-table insert by @SkyFan2002 in #15002
  • feat(query): to_timestamp support parser string to ts without timezone by @TCeason in #15131
  • feat: Enhance show views by @blackstar-baba in #15109
  • feat: support expr in values clause of multi table insert by @SkyFan2002 in #15147
  • feat(query): implement ST_GEOMFROMGEOHASH by @kkk25641463 in #15156
  • feat(query): show create table support display inverted index by @b41sh in #15169
  • feat(query): support CHANGES clause by @zhyass in #15163
  • feat: add inverted index pruning status by @BohuTANG in #15175
  • feat: Support to add random seed on random engine by @blackstar-baba in #15167
  • feat(planner): push down not in filter expression by @Dousir9 in #15168
  • feat: time travel support at offset by @zhyass in #15177

Thoughtful Bug Fix 🔧

  • fix(ci): fix internal benchmark by @TCeason in #14897
  • fix: Display of merge into .. insert * by @dantengsky in #14892
  • fix: check_and_upgrade_to_pb() should return MetaError, add quota by @drmingdrmer in #14899
  • fix: Display of compact AST by @SkyFan2002 in #14914
  • fix(query): fix native topk_sorter by @sundy-li in #14939
  • fix: stream source can't get the same tablemeta by @zhyass in #14935
  • fix: cluster table with change tracking enabled append panic by @zhyass in #14956
  • fix: enable runtime filter when join_spilling_memory_ratio !=0 by @xudong963 in #14959
  • fix: fix stream explain bug for merge into by @JackTan25 in #14969
  • fix(query): Need return error udf_name when derialize from pb failed. by @TCeason in #14964
  • fix: try fix compute stddev for decimal type overflow by @ariesdevil in #14952
  • fix: csv read error with skip_header. by @youngsofun in #14981
  • fix: fix incorrect agg spill in new agg hashtable by @Freejww in #14995
  • fix: wrong row id by @youngsofun in #15018
  • fix: trim leading '/'s for files specified in copy opt by @dantengsky in #15056
  • fix: stream get wrong result if change tracking is changed by @zhyass in #15072
  • fix(ast): fix incorrect display for ast (part 1) by @andylokandy in #15069
  • fix: privilege check for insert statement by @SkyFan2002 in #15075
  • fix(query): fix attach_query_str by @sundy-li in #15066
  • fix: fix plan_sql error can not write query_log by @BohuTANG in #15077
  • fix: improve error hints when select * from xx.yy by @ZhengLin-Li in #14663
  • fix: Support udf for copy table by @JackTan25 in #15078
  • fix(ast): fix incorrect display for ast (part 2) by @andylokandy in #15076
  • fix: make block data type be consistent with hash method by @xudong963 in #15089
  • fix: add prql description by @ncuwaln in #15091
  • fix: sqllogicaltest datatype mismatch in expr_in_window.test by @c8ef in #15090
  • fix: schema mismatch in named_window_basic.test by @c8ef in #15099
  • fix: test duplicate in window_basic.test by @c8ef in #15100
  • fix(query): verify_privilege need check privilege one by one by @TCeason in #15105
  • fix(query):user option DEFAULT_ROLE ignore ascii_case by @TCeason in #15137
  • fix(query): tuple field name can not contain special char by @TCeason in #15126
  • fix(query): new agg hashtable hung with little data on singleton by @Freejww in #15140
  • fix(query): show query with name consider quote by @TCeason in #15138
  • fix: forbid end at stream syntax by @zhyass in #15173
  • fix(storage): Don't return empty credential while loading failed by @Xuanwo in #15181

Code Refactor 🎉

  • refactor: upgrade to openraft 0.9.0 by @drmingdrmer in #14907
  • refactor: use kvapi::Key to define meta-service key for QuotaMgr by @drmingdrmer in #14915
  • refactor: reduce hash join build spilling files by @xudong963 in #14919
  • refactor: define kvapi::Key for PasswordPolicyMgr by @drmingdrmer in #14920
  • refactor: reduce meta call by @TCeason in #14917
  • refactor: add NetworkPolicyIdent as NetworkPolicy meta-service key by @drmingdrmer in #14933
  • refactor: http handler plan sql async. by @youngsofun in #14908
  • refactor: Add TIdent for all meta-service key that share the same form of <PREFIX>/<TENANT>/<NAME> by @drmingdrmer in #14940
  • refactor: add generic CrudMgr to implement various simple CRUD manager by @drmingdrmer in #14988
  • refactor(query): use project plan for copy into select by @zhang2014 in #14977
  • refactor: use CrudMgr to impl PasswordPolicyMgr by @drmingdrmer in #14994
  • refactor: Add ExistError and UnknownError, TenantResource does not need to define error type any more by @drmingdrmer in #14996
  • refactor: Use CrudMgr to impl FileFormatMgr by @drmingdrmer in #14999
  • refactor: Use CrudMgr to impl ConnectionMgr by @drmingdrmer in #15003
  • refactor(query): add copy into location physical plan by @zhang2014 in #15010
  • refactor(query): csv reader support prefetch by @youngsofun in #14983
  • refactor: alter view only need to modify table option by @TCeason in #15012
  • refactor: truncate support build pipeline by @zhyass in #15027
  • refactor: move id-generator from crate meta-api to meta-app by @drmingdrmer in #15036
  • refactor: remove innecessary serde by @drmingdrmer in #15037
  • refactor(query): use scan table physical plan for copy into table from stage by @zhang2014 in #15016
  • refactor: remove innecessary serde by @drmingdrmer in #15040
  • refactor: remove innecessary serde by @drmingdrmer in #15042
  • refactor: spead up csv reader. by @youngsofun in #15043
  • refactor(query): remove metrics if moved to the profile by @zhang2014 in #15054
  • refactor: reduce hash join probe spilled files by @xudong963 in #14942
  • refactor: separate key and value: add BackgroundTaskCreator by @drmingdrmer in #15063
  • refactor: remove serde from CatalogId and CatalogNameIdent by @drmingdrmer in #15068
  • refactor: remove serde from kvapi::Key types by @drmingdrmer in #15070
  • refactor: use type Tenant for BackgroundJobIdent.tenant by @drmingdrmer in #15074
  • refactor: BackgroundTaskIdent use Tenant by @drmingdrmer in #15080
  • refactor: make the block pruning more clear by @BohuTANG in #15079
  • refactor: change PartInfo is_lazy to LazyLevel by @BohuTANG in #15081
  • refactor: QueryFragmentsPlanPacket should include just setting changes by @drmingdrmer in #15095
  • refactor(executor): refactor queries executor to prepare by @dqhl76 in #14965
  • refactor: show grants support display ObjectId by @TCeason in #15064
  • refactor: replace NonEmptyString with Tenant by @drmingdrmer in #15103
  • refactor: replace String with Tenant by @drmingdrmer in #15119
  • refactor: use TIdent to unify kvapi key def for MaskPolicyTableIdListIdent and DataMaskNameIdent by @drmingdrmer in #15123
  • refactor: remove count-table for tenant. It is not used at all by @drmingdrmer in #15134
  • refactor: use Tenat in VirtualColumnNameIdent by @drmingdrmer in #15135
  • refactor(query): record physical plan level metrics into profiling log by @zhang2014 in #15092
  • refactor: define ShareEndpointIdent with TIdent by @drmingdrmer in #15136
  • refactor: add peer info to network error context by @drmingdrmer in #15139
  • refactor: add Tenant to ShareConsumer by @drmingdrmer in #15142
  • chore: client should not depend on the final response. by @youngsofun in #15146
  • refactor(executor): globalize QueriesPipelineExecutor creation method by @dqhl76 in #15129
  • refactor(planner): remove statistics collection from bind phase by @Dousir9 in #15112
  • refactor(query): only intercept dml and queries in queries queue by @zhang2014 in #15143
  • refactor: user can execute show grants query itself by @TCeason in #15149
  • refactor: generate inverted indexs for each blocks by @b41sh in #15150
  • refactor: remove dependency on parquet2: Part I by @SkyFan2002 in #15158
  • refactor: use Tenant for ListBackgroundTasksReq by @drmingdrmer in #15176
  • refactor: move TIdent into separate files by @drmingdrmer in #15179

Others 📒

  • chore(storage): split mutation_log file by @zhyass in #14903
  • chore(query): create user/role name can not contain single/double quote by @TCeason in #14902
  • chore(ci): benchmark local need set meta client timeout by @TCeason in #14910
  • chore: extend time tolerance for meta-service test by @drmingdrmer in #14927
  • chore: add step logs for hash join build side by @xudong963 in #14930
  • chore: use shorter name for management API by @drmingdrmer in #14937
  • chore(ci): add pr summary check by @everpcpc in #14944
  • chore(ci): fix set failure for pr check by @everpcpc in #14946
  • chore(ci): fix description check by @everpcpc in #14947
  • chore(ci): fix event for pr assistant by @everpcpc in #14948
  • chore(query): improve push_column_with_selection by @Dousir9 in #14954
  • chore(query): add disable system tables by @zhang2014 in #14958
  • chore(planner): fix distributed query plan by @Dousir9 in #14951
  • chore: tweak txn syntax by @SkyFan2002 in #14960
  • chore(query): add settings level for admin api by @zhang2014 in #14967
  • chore(ast): CreatOption::None -> CreateOption::Create by @andylokandy in #14941
  • chore: improve histogram‘s implement and comments by @suimenno3002 in #14976
  • chore: number range for settings by @BohuTANG in #14985
  • chore: group limit optimization in new hashtable by @sundy-li in #14989
  • chore: add integration test for runtime filter by @xudong963 in #14992
  • chore: upgrade openraft to 0.9.1 by @drmingdrmer in #15006
  • chore: max_running_queries from 8 to 0, disable the max running queries by @BohuTANG in #15013
  • chore: tweak deletion batch size & log messages by @dantengsky in #15005
  • chore: rollback partial tests by @JackTan25 in #15015
  • chore: add benchmark for loading data on cloud by @everpcpc in #15017
  • chore: add a setting for inlist to join by @xudong963 in #15024
  • chore: unify TIdent key format by @drmingdrmer in #15025
  • chore: the fist http resp wait longer to tolerant old bendsql clients. by @youngsofun in #15022
  • chore: add log about presign elapsed by @flaneur2020 in #14990
  • chore: fix sql seq for benchmark report by @everpcpc in #15030
  • chore: print query detail for bench & fix prepare database for load bench by @everpcpc in #15032
  • chore: default enalbe merge into by @BohuTANG in #15034
  • chore: run load test with new label by @everpcpc in #15035
  • chore(query): add spill profile for join by @zhang2014 in #15044
  • chore(http): tune log on getting /v1/query/:id/kill by @flaneur2020 in #14987
  • chore: default enable aggregate/sort/join spill by @BohuTANG in #15038
  • chore: do not drop database first for bench by @everpcpc in #15041
  • chore(query): improve group by @sundy-li in #15031
  • chore(planner): fix invalid aggregate physical plan by @Dousir9 in #15046
  • chore: adjust the default value for table lock timeout by @zhyass in #15055
  • chore: use Tenant::new_literal() and new_nonempty() to avoid creating empty Tenant by @drmingdrmer in #15061
  • chore(ci): fix benchmark job dependency by @everpcpc in #15087
  • chore: remove rebinding merge into for insert-only by @xudong963 in #15086
  • chore: clean z3 and meta dir by @zhyass in #15098
  • chore(executor): remove duplicate profile statistics by @zhang2014 in #15115
  • chore(query): errorcode use snake shape format by @TCeason in #15122
  • chore: treat seq as place holder when navigate by @zhyass in #15124
  • chore(planner): make optimize function async by @Dousir9 in #15121
  • chore: adjust stack size of flaky ut by @SkyFan2002 in #15130
  • chore: to avoid UT stackoverflow, disable async_backtrace of TypeChecker::resolve by @dantengsky in #15133
  • chore(query): refactor new transform partition bucket for new aggregation hashtable by @Freejww in #15093
  • chore: enable inlist to join in cluster by @xudong963 in #15108
  • chore: remove no longer used message encoding benchmark by @dantengsky in #15148
  • chore(query): fix order for scalar cmp with null by @sundy-li in #15145
  • chore: fix typos by @Dousir9 in #15153
  • chore(query): use i64 min max value as histogram bucket min max limit by @zhang2014 in #15152
  • chore(query): turn on new agg hashtable by @Freejww in #15155
  • chore(query): row-fetcher-cache by @sundy-li in #15159
  • chore: add CopyInto to query queue by @BohuTANG in #15160
  • chore: refine the query/02_function to query/functions by @BohuTANG in #15161
  • chore: refine the query queue is_heavy_action func by @BohuTANG in #15166
  • chore: fix window plan in lazy topn by @sundy-li in #15165
  • chore(deps): bump h2 from 0.3.24 to 0.3.26 by @dependabot in #15172
  • chore(query): fix boolean type comparison by @Dousir9 in #15171
  • chore: return inserted rows for each table in multi table insert by @SkyFan2002 in #15178

New Contributors

  • @suimenno3002 made their first contribution in #14839
  • @ncuwaln made their first contribution in #14922
  • @blackstar-baba made their first contribution in #14926
  • @kkk25641463 made their first contribution in #15059
  • @c8ef made their first contribution in #15090

Full Changelog: https://github.com/datafuselabs/databend/compare/v1.2.371...v1.2.410

Mar 11, 2024 (v1.2.371)

What's Changed

Exciting New Features ✨

  • feat: add div0 divnull method by @sundy-li in #14693
  • feat(query): grant object visibility check by @TCeason in #14458
  • feat: add a new kvapi::Key type: Tenant by @drmingdrmer in #14704
  • feat(query): privilege access check need consider if exists by @TCeason in #14700
  • feat: add kvapi::Value to define meta-service value behavior by @drmingdrmer in #14708
  • feat(geo): support geometry type for create table by @ariesdevil in #14615
  • feat(query): support vacuum temporary files by @zhang2014 in #14690
  • feat: add parent() method to kvapi::Key to describe the hierarchical structure of meta-service data by @drmingdrmer in #14716
  • feat(query): integrate hash join with new filter framework by @Dousir9 in #14689
  • feat: add kvapi::ValueType::dependency_keys() to retrieve relation such as table-name-to-table-id and table-id-to-table-meta by @drmingdrmer in #14735
  • feat: support multi-statement transactions by @SkyFan2002 in #14562
  • feat: DESC[RIBE][TABLE] name by @dracoooooo in #14736
  • feat(query): new implementation of analyze table by @sundy-li in #14725
  • feat: add Analyze for merge into by @JackTan25 in #14763
  • feat(executor): implement time slicing for shared executor scheduling across queries by @dqhl76 in #14770
  • feat(sqlsmith): Support generating binary and geometry values by @b41sh in #14758
  • feat: get_client_info() returns the server time in milliseconds by @drmingdrmer in #14805
  • feat: http handler txn support. by @youngsofun in #14802
  • feat(planner): implement explain query in verbose mode by @leiysky in #14801
  • feat: add new string view type [part 1] by @ariesdevil in #14662
  • feat: upgrade arrow to version 50 by @sundy-li in #14784
  • feat(query): introduce arrow-udf-js by @sundy-li in #14799
  • feat: support left-related join spilling by @xudong963 in #14828
  • feat: impl like selectivity compution by probability predication by @xudong963 in #14846
  • feat: add vacuum table result table by @lichuang in #14830
  • feat(query): read write inverted index by @b41sh in #14827
  • feat: change tracking enabled table support replace into by @zhyass in #14831
  • feat: support notification syntax with rpc, and support task error notification integration by @ZhiHanZ in #14845
  • feat: support right-related join spilling by @xudong963 in #14853
  • feat: add vacuum table dry run result table by @lichuang in #14851
  • feat: add transaction and mutli-line SQL script support for task by @ZhiHanZ in #14865
  • feat: add vacuum drop table dry run result table by @lichuang in #14870
  • feat(query): support variant is_type functions by @b41sh in #14876
  • feat(query): support queries queue by @zhang2014 in #14878
  • feat(query): create drop inverted index by @b41sh in #14859
  • feat(query): show grants support mget table/db name by table/db id. by @TCeason in #14862

Thoughtful Bug Fix 🔧

  • fix: do compact first , then do sort for cluster table by @JackTan25 in #14707
  • fix(query): filter response session settings with ScopeLevel::Session by @everpcpc in #14702
  • fix: fix create or replace table as select bug by @lichuang in #14730
  • fix: check if global memory is negative by @xudong963 in #14733
  • fix: Don't return empty credentail if load failed by @Xuanwo in #14734
  • fix: wrong result for CSV EMPTY_FIELD_AS = STRING. by @youngsofun in #14744
  • fix: fix create or replace stage bug by @lichuang in #14745
  • fix: force attachment to use EmptyFieldAs::FieldDefault. by @youngsofun in #14767
  • fix: fix create or replace stage bug, only remove dir when success by @lichuang in #14766
  • fix(planner): use correct alias for column reference in select clause by @leiysky in #14775
  • fix(executor): queries pipeline executor schedule incorrectly by @dqhl76 in #14787
  • fix: fix hash join spilling hang and add more tests by @xudong963 in #14794
  • fix(ast): add missing #[drive(skip)] by @andylokandy in #14823
  • fix: Transaction state is not changed correctly when returning error in commit statement by @SkyFan2002 in #14824
  • fix(query): today, yesterday, tomorrow need consider tz by @TCeason in #14841
  • fix: flaky sqllogic test by @zhyass in #14848
  • fix: fix wrong results for left join spilling by @xudong963 in #14860
  • fix: allow insert in multiline statment by @andylokandy in #14882
  • fix: rewrite ifnull and nvl as coalesce by @PsiACE in #14877
  • fix: fix incorrect schema in window expr in subquery with lazy_materialization by @ariesdevil in #14895

Code Refactor 🎉

  • refactor: new impl for loading CSV. by @youngsofun in #14645
  • refactor: add UdfError::Exist by @drmingdrmer in #14691
  • refactor: remove kvapi::Key prefix const by @drmingdrmer in #14701
  • refactor: add Humanizer traits for operator display by @leiysky in #14718
  • refactor: schema_api::update_mask_policy by @drmingdrmer in #14724