Supeset save query api returns 500 internal error post db upgrade

[maclynnbrandlock]

Bug description

https://{hosturl}/api/v1/saved_query/?q=(order_column:changed_on_delta_humanized,order_direction:desc,page:0,page_size:25)

Supeset save query api returns 500 internal error post db upgrade

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[maclynnbrandlock]

superset db upgrade
/home/ec2-user/superset/venv/lib64/python3.9/site-packages/superset/config.py:42: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
import pkg_resources
WARNI [alembic.env] SQLite Database support for metadata databases will be removed in a future version of Superset.
INFO [alembic.env] Starting the migration scripts.
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 4a33124c18ad -> 58d051681a3b, Add catalog_perm to tables
Column 'catalog_perm' already exists in table 'tables'.

Column 'catalog_perm' already exists in table 'slices'.

Traceback (most recent call last):
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 126, in decrypt
decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdb in position 1: invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ec2-user/superset/venv/bin/superset", line 6, in
sys.exit(superset())
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/flask/cli.py", line 358, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/flask_migrate/cli.py", line 149, in upgrade
_upgrade(directory, revision, sql, tag, x_arg)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/flask_migrate/init.py", line 98, in wrapped
f(*args, **kwargs)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/flask_migrate/init.py", line 185, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/alembic/command.py", line 403, in upgrade
script.run_env()
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/alembic/script/base.py", line 583, in run_env
util.load_python_file(self.dir, "env.py")
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
module = load_module_py(module_id, path)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/superset/extensions/../migrations/env.py", line 142, in
run_migrations_online()
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/superset/extensions/../migrations/env.py", line 133, in run_migrations_online
context.run_migrations()
File "", line 8, in run_migrations
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
self.get_context().run_migrations(**kw)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/alembic/runtime/migration.py", line 627, in run_migrations
step.migration_fn(**kw)
File "/home/ec2-user/superset/venv/lib/python3.9/site-packages/superset/migrations/versions/2024-05-01_10-52_58d051681a3b_add_catalog_perm_to_tables.py", line 48, in upgrade
upgrade_catalog_perms(engines={"postgresql"})
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/superset/migrations/shared/catalogs.py", line 374, in upgrade_catalog_perms
for database in session.query(Database).all():
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy/orm/query.py", line 2773, in all
return self._iter().all()
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy/engine/result.py", line 1476, in all
return self._allrows()
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy/engine/result.py", line 401, in _allrows
rows = self._fetchall_impl()
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy/engine/result.py", line 1389, in _fetchall_impl
return self._real_result._fetchall_impl()
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy/engine/result.py", line 1813, in _fetchall_impl
return list(self.iterator)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy/orm/loading.py", line 147, in chunks
fetch = cursor._raw_all_rows()
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy/engine/result.py", line 393, in _raw_all_rows
return [make_row(row) for row in rows]
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy/engine/result.py", line 393, in
return [make_row(row) for row in rows]
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1716, in process
return process_value(impl_processor(value), dialect)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 479, in process_result_value
value = super().process_result_value(value=value, dialect=dialect)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 424, in process_result_value
decrypted_value = self.engine.decrypt(value)
File "/home/ec2-user/superset/venv/lib64/python3.9/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt
raise ValueError('Invalid decryption key')

[dosubot]

Hi @maclynnbrandlock! I'm Dosu and I’m helping the superset team.

The Invalid decryption key error indicates your current SUPERSET_SECRET_KEY doesn't match the key originally used to encrypt your database credentials. The migration 58d051681a3b_add_catalog_perm_to_tables queries all database connections, which triggers decryption of the encrypted password, encrypted_extra, and server_cert fields.

If you have the original SECRET_KEY, use the key rotation process:

export PREVIOUS_SECRET_KEY=<your_old_key>
export SUPERSET_SECRET_KEY=<your_new_key>
superset re-encrypt-secrets
superset db upgrade

If you've lost the original key, you'll need to clear the encrypted fields and re-enter credentials after migration:

UPDATE dbs SET password = NULL, encrypted_extra = NULL;

Also ensure SUPERSET_SECRET_KEY is set correctly in your environment before running migrations — this environment variable takes precedence over config file settings.

This is a known issue when upgrading Superset, especially if the SECRET_KEY changed or wasn't consistently set across environments.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[saks610]

GET /api/v1/saved_query/?q=(order_column:changed_on_delta_humanized,order_direction:desc,page:0,page_size:25)

Observations

The issue occurs when ordering by changed_on_delta_humanized.

This appears to be a computed/derived property rather than a physical database column.

After running superset db upgrade, the ORM may be attempting to apply SQL-level ordering on this non-existent column.

This likely results in a SQLAlchemy error, which causes the 500 response.

Verification

All migrations were executed using:

superset db upgrade

Temporary Workaround

Changing the request to:

q=(order_column:changed_on,order_direction:desc,page:0,page_size:25)

works correctly and avoids the 500 error.

This suggests the issue is related to sorting on a derived/computed field.

Happy to provide logs or help test a fix if needed.