Merge branch 'develop' into user-type

Author: zackkrida

.gitignore

@@ -181,6 +181,7 @@ Temporary Items
 .media/
 .secrets/
 docs/_build/
+msar/
 
 ## yaml config files ##
 sso.yml

CONTRIBUTING.md

@@ -6,13 +6,13 @@ Mathesar's development happens on [GitHub](https://github.com/mathesar-foundatio
 
 ## Joining the Community
 
-We highly recommend joining our [Matrix community](https://wiki.mathesar.org/en/community/matrix) and our [developer mailing list](https://wiki.mathesar.org/en/community/mailing-lists) before making contributions. This is where most of the core team's conversations about building Mathesar happen.
+We highly recommend joining our [Matrix community](https://wiki.mathesar.org/community/matrix) and our [developer mailing list](https://wiki.mathesar.org/community/mailing-lists) before making contributions. This is where most of the core team's conversations about building Mathesar happen.
 
 ## Contributing code
 
 1. **Get Mathesar [running locally](./DEVELOPER_GUIDE.md#local-development-setup).**
 
-    Make sure to **do this before moving on**. If you need help, ask in [Matrix](https://wiki.mathesar.org/en/community/matrix), taking care to form *specific* questions that people can answer asynchronously.
+    Make sure to **do this before moving on**. If you need help, ask in [Matrix](https://wiki.mathesar.org/community/matrix), taking care to form *specific* questions that people can answer asynchronously.
 
 1. **Find an [issue](https://github.com/mathesar-foundation/mathesar/issues) to work on.**
 
@@ -21,7 +21,7 @@ We highly recommend joining our [Matrix community](https://wiki.mathesar.org/en/
     - ❌ If an issue is _not_ labeled "help wanted", then it is not open to community contribution. One of the Mathesar maintainers will work on it instead.
     - ❌ Issues already assigned to other users are also not open to contribution.
 
-    If you want to work on something for which there is no GitHub issue open yet, [create an issue](https://github.com/mathesar-foundation/mathesar/issues/new/choose) and propose your change there. A Mathesar [team member](https://wiki.mathesar.org/en/team) will evaluate your issue and decide whether we'll accept a pull request for the issue.
+    If you want to work on something for which there is no GitHub issue open yet, [create an issue](https://github.com/mathesar-foundation/mathesar/issues/new/choose) and propose your change there. A Mathesar [team member](https://wiki.mathesar.org/team) will evaluate your issue and decide whether we'll accept a pull request for the issue.
 
 1. ***(Optionally)* Claim the issue.**
 
@@ -56,7 +56,7 @@ We highly recommend joining our [Matrix community](https://wiki.mathesar.org/en/
 
 ## Contributing PR reviews
 
-We encourage and appreciate code review by contributors. Feel free to review any open pull requests. Follow our [code review guidelines](https://wiki.mathesar.org/en/engineering/code-review).
+We encourage and appreciate code review by contributors. Feel free to review any open pull requests. Follow our [code review guidelines](https://wiki.mathesar.org/engineering/code-review).
 
 ## Contributing documentation
 
@@ -68,5 +68,5 @@ We encourage and appreciate code review by contributors. Feel free to review any
 
 Due to limited capacity, we are currently unable to accept design volunteers. Please return to this page for updates.
 
-Please read through our [Design](https://wiki.mathesar.org/en/design) section to learn more about our design process.
+Please read through our [Design](https://wiki.mathesar.org/archive/product/design/) section to learn more about our design process.
 

Caddyfile

@@ -12,7 +12,7 @@
 
 		file_server {
 			precompressed br zstd gzip
-			root {$MEDIA_ROOT:/code/media/}
+			root {$MEDIA_ROOT:/code/.media/}
 		}
 	}
 	handle_path /static/* {

README.md

@@ -12,8 +12,8 @@
   <a href="https://mathesar.org?ref=github-readme" target="_blank">Website</a> • <a href="https://docs.mathesar.org?ref=github-readme-top" target="_blank">Docs</a> • <a href="https://wiki.mathesar.org/en/community/matrix" target="_blank">Matrix (chat)</a> • <a href="https://discord.gg/enaKqGn5xx" target="_blank">Discord</a> • <a href="https://wiki.mathesar.org/" target="_blank">Contributor Wiki</a> • <a href="https://github.com/orgs/mathesar-foundation/projects/2" target="_blank">Roadmap</a>
 </p>
 
-
 # Mathesar
+
 Mathesar is a web application that makes working with PostgreSQL databases both simple and powerful. It empowers users of all technical skill levels to view, edit, query, and collaborate on data with a familiar spreadsheet-like interface—no code needed. It’s self hosted, can be deployed in minutes, and works directly with PostgreSQL databases, schemas, and tables without extra abstractions. The project is 100% open source and maintained by Mathesar Foundation, a 501(c)(3) nonprofit.
 
 Mathesar is as scalable as Postgres and supports any size or complexity of data, making it ideal for workflows involving production databases. It requires minimal setup, and integrates into your existing infrastructure. Because Mathesar is self-hosted, your data never leaves your servers, and access control based on Postgres roles and privileges keeps your database secure without adding unnecessary risk.
@@ -36,6 +36,8 @@ Mathesar is as scalable as Postgres and supports any size or complexity of data,
   - [Linking two tables together](#linking-two-tables-together)
   - [Managing table permissions](#managing-table-permissions)
   - [Viewing a single record with related records](#viewing-a-single-record-with-related-records)
+  - [Building a form](#building-a-form)
+  - [Filling out a form](#filling-out-a-form)
   - [Viewing an Exploration (query result)](#viewing-an-exploration-query-result)
   - [Building an Exploration (query)](#building-an-exploration-query)
   - [Disconnecting a database](#disconnecting-a-database)
@@ -47,22 +49,27 @@ Mathesar is as scalable as Postgres and supports any size or complexity of data,
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
 ## Status
+
 - [x] **Public Alpha**: You can install and deploy Mathesar on your server. Go easy on us!
 - [x] **Public Beta**: Stable and feature-rich enough to implement in production
 - [ ] **Public**: Widely used in production environments
 
 We are currently in the **public beta** stage.
 
 ## Install Mathesar
+
 Please see [our documentation](https://docs.mathesar.org/?ref=github-readme-installing) for instructions on installing Mathesar on your own server.
 
 ## Join our community
+
 The Mathesar team is on [Matrix](https://wiki.mathesar.org/en/community/matrix) (chat service). We also have [mailing lists](https://wiki.mathesar.org/en/community/mailing-lists) and the core team discusses day-to-day work on our developer mailing list.
 
 ### Contribute to Mathesar
+
 We actively encourage contribution! Get started by reading our [Contributor Guide](./CONTRIBUTING.md).
 
 ## Features
+
 - **Built on Postgres**: Connect to an existing Postgres database or set one up from scratch.
 - **Install in minutes**: Install using Docker in minutes, integrate into any existing infrastructure.
 - **Postgres-based access control**: Use existing Postgres roles within Mathesar's UI, or set up your own.
@@ -71,13 +78,16 @@ We actively encourage contribution! Get started by reading our [Contributor Guid
 - **Data entry**: Use our spreadsheet-like interface to view, create, update, and delete table records.
 - **Filter, sort, and group**: Quickly slice your data in different ways.
 - **Query builder**: Use our Data Explorer to build queries without knowing anything about SQL or joins.
+- **Forms**: Build and share forms to collect data from anyone via a unique link, with submissions automatically saved as new records.
 - **Import and export data**: Import and export data into Mathesar easily to work with your data elsewhere.
 - **Schema migrations**: Transfer columns between tables in two clicks.
 - **Uses Postgres features**: Mathesar uses and manipulates Postgres schemas, primary keys, foreign keys, constraints and data types. e.g. "Relationships" in the UI are foreign keys in the database.
 - **Custom data types**: Custom data types for emails and URLs, validated at the database level.
 
 ## Screenshots
+
 ### Connecting a database
+
 ![connect-db](https://github.com/user-attachments/assets/d7188c8d-a040-4bc4-8350-28b2a6fd1661)
 
 ### Adding collaborators
@@ -101,6 +111,12 @@ We actively encourage contribution! Get started by reading our [Contributor Guid
 ### Viewing a single record with related records
 ![record-page](https://github.com/user-attachments/assets/d2e621e3-5597-44ef-ae87-482bec0b9bcc)
 
+### Building a form
+![form-builder](https://raw.githubusercontent.com/mathesar-foundation/mathesar/develop/docs/docs/assets/releases/0.5.0/form-builder.png)
+
+### Filling out a form
+![form-fill-example](https://raw.githubusercontent.com/mathesar-foundation/mathesar/develop/docs/docs/assets/releases/0.5.0/form-fill-example.png)
+
 ### Viewing an Exploration (query result)
 ![viewing-exploration](https://github.com/user-attachments/assets/410cb99f-a583-411b-9706-7967bff673e6)
 
@@ -111,15 +127,19 @@ We actively encourage contribution! Get started by reading our [Contributor Guid
 ![disconnect-db](https://github.com/user-attachments/assets/f22e9c78-3c92-40c8-8ef9-dcb4cf8e8aea)
 
 ## Mathesar in action
+
 https://github.com/user-attachments/assets/6bdfb178-17b4-4abf-aac4-9781e1d841ab
 
 ## Our motivation
+
 Using databases shouldn't require technical expertise or expensive, closed-off tools. Databases are incredibly powerful, but they're often trapped behind complex interfaces that are hard to use or limit how people can access and share their data. We want to change that by building user-friendly tools that unlock the power of existing databases without sacrificing accessibility, portability, or extensibility.
 
 Mathesar is our answer: an open-source platform designed to unlock the full potential of PostgreSQL, one of the most powerful and trusted open-source databases. Mathesar is easy to use, interoperable, and extensible, while also giving you complete control over your data. As a nonprofit, we're committed to keeping Mathesar 100% open source and available to everyone—because better ways to work with data mean better decisions, and better decisions lead to a better world.
 
 ## Bugs and troubleshooting
+
 If you run into problems, refer to our [troubleshooting guide](./TROUBLESHOOTING.md).
 
 ## License
+
 Mathesar is open source under the GPLv3 license - see [LICENSE](LICENSE). It also contains derivatives of third-party open source modules licensed under the MIT license. See the list and respective licenses in [THIRDPARTY](THIRDPARTY).

config/database_config.py

@@ -99,7 +99,7 @@ def to_django_dict(self) -> Dict[str, Any]:
 @dataclass(frozen=True)
 class PostgresConfig(DBConfig):
     engine: str = POSTGRES_ENGINE
-    sslmode: Optional[str] = None
+    sslmode: str = "prefer"
 
     # Inject sslmode into OPTIONS
     # https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-PROTECTION
@@ -115,7 +115,9 @@ def from_connection_string(cls, url: str) -> "PostgresConfig":
         params = conninfo.conninfo_to_dict(url)
         dbname = params.get("dbname")
         if not dbname:
-            raise ValueError("PostgresConfig.from_connection_string: missing database name in URL")
+            raise ValueError(
+                "PostgresConfig.from_connection_string: missing database name in URL"
+            )
 
         return cls(
             dbname=dbname,
@@ -129,7 +131,7 @@ def from_connection_string(cls, url: str) -> "PostgresConfig":
     @classmethod
     def from_django_dict(cls, cfg: Mapping[str, Any]) -> "PostgresConfig":
         raw_opts = cfg.get("OPTIONS", {}).copy()
-        sslmode = raw_opts.pop("sslmode", None)
+        sslmode = raw_opts.pop("sslmode", "prefer")
         base_cfg = dict(cfg, OPTIONS=raw_opts)
         base = super().from_django_dict(base_cfg)
         return replace(base, sslmode=sslmode)

config/settings/common_settings.py

@@ -35,9 +35,6 @@
     "django.contrib.messages",
     "whitenoise.runserver_nostatic",
     "django.contrib.staticfiles",
-    "rest_framework",
-    "django_filters",
-    "django_property_filter",
     "modernrpc",
     "mathesar",
     "allauth",
@@ -208,6 +205,7 @@
 POSTGRES_PASSWORD = os.environ.get('POSTGRES_PASSWORD', default=None)
 POSTGRES_HOST = os.environ.get('POSTGRES_HOST', default=None)
 POSTGRES_PORT = os.environ.get('POSTGRES_PORT', default=None)
+POSTGRES_SSLMODE = os.environ.get('POSTGRES_SSLMODE', default='prefer')
 
 # POSTGRES_DB, POSTGRES_USER, and POSTGRES_HOST are required env variables for forming a pg connection string for the django database
 if POSTGRES_DB and POSTGRES_USER and POSTGRES_HOST:
@@ -217,6 +215,7 @@
         port=parse_port(POSTGRES_PORT),
         role=POSTGRES_USER,
         password=POSTGRES_PASSWORD,
+        sslmode=POSTGRES_SSLMODE,
     ).to_django_dict()
     DATABASES['default']['OPTIONS'] = {
         "application_name": "Mathesar Django"
@@ -302,35 +301,12 @@
 
 MEDIA_URL = "/media/"
 
-# Update Authentication classes, removed BasicAuthentication
-# Defaults: https://www.django-rest-framework.org/api-guide/settings/
-REST_FRAMEWORK = {
-    'DEFAULT_AUTHENTICATION_CLASSES': [
-        'rest_framework.authentication.TokenAuthentication',
-        'rest_framework.authentication.SessionAuthentication'
-    ],
-    'DEFAULT_PERMISSION_CLASSES': [
-        'rest_framework.permissions.IsAuthenticated',
-    ],
-    'DEFAULT_PARSER_CLASSES': [
-        'rest_framework.parsers.JSONParser',
-    ],
-    'DEFAULT_FILTER_BACKENDS': (
-        'django_filters.rest_framework.DjangoFilterBackend',
-        'rest_framework.filters.OrderingFilter',
-    ),
-    'TEST_REQUEST_DEFAULT_FORMAT': 'json',
-    'EXCEPTION_HANDLER':
-        'mathesar.exception_handlers.mathesar_exception_handler',
-}
-
 # Mathesar settings
 MATHESAR_MODE = os.environ.get('MODE', default='PRODUCTION')
 MATHESAR_UI_BUILD_LOCATION = os.path.join(BASE_DIR, 'mathesar/static/mathesar/')
 MATHESAR_MANIFEST_LOCATION = os.path.join(MATHESAR_UI_BUILD_LOCATION, 'manifest.json')
 MATHESAR_CLIENT_DEV_PORT = os.environ.get('MATHESAR_CLIENT_DEV_PORT', default='3000')
 MATHESAR_UI_SOURCE_LOCATION = os.path.join(BASE_DIR, 'mathesar_ui/')
-MATHESAR_CAPTURE_UNHANDLED_EXCEPTION = os.environ.get('CAPTURE_UNHANDLED_EXCEPTION', default=False)
 MATHESAR_STATIC_NON_CODE_FILES_LOCATION = os.path.join(BASE_DIR, 'mathesar/static/non-code/')
 MATHESAR_ANALYTICS_URL = os.environ.get('MATHESAR_ANALYTICS_URL', default='https://example.com/collector')
 MATHESAR_INIT_REPORT_URL = os.environ.get('MATHESAR_INIT_REPORT_URL', default='https://example.com/hello')
@@ -349,9 +325,7 @@
 LOGIN_URL = '/auth/login/'
 LOGIN_REDIRECT_URL = '/'
 LOGOUT_REDIRECT_URL = LOGIN_URL
-DRF_ACCESS_POLICY = {
-    'reusable_conditions': ['mathesar.api.permission_conditions']
-}
+
 # List of Template names that contains additional script tags to be added to the base template
 BASE_TEMPLATE_ADDITIONAL_SCRIPT_TEMPLATES = []
 

config/urls.py

@@ -22,7 +22,6 @@
 urlpatterns = [
     path('admin/', admin.site.urls),
     path('', include(mathesar_urls)),
-    path('api-docs/', include('rest_framework.urls')),
 ]
 
 handler404 = 'mathesar.views.page_not_found_view'

db/identifiers.py

@@ -33,13 +33,8 @@ def truncate_if_necessary(identifier):
 
 
 def is_identifier_too_long(identifier):
-    # TODO we should support POSTGRES_IDENTIFIER_SIZE_LIMIT here;
-    # Our current limit due to an unknown bug that manifests at least
-    # when importing CSVs seems to be 57 bytes. Here we're setting it even
-    # lower just in case.
-    our_temporary_identifier_size_limit = 48
     size = _get_size_of_identifier_in_bytes(identifier)
-    return size > our_temporary_identifier_size_limit
+    return size > POSTGRES_IDENTIFIER_SIZE_LIMIT
 
 
 def _get_truncation_hash(identifier):

db/records.py

@@ -15,6 +15,7 @@ def list_records_from_table(
     order=None,
     filter=None,
     group=None,
+    joined_columns=None,
     return_record_summaries=False,
     table_record_summary_templates=None,
 ):
@@ -34,6 +35,8 @@ def list_records_from_table(
         order: An array of ordering definition objects.
         filter: An array of filter definition objects.
         group: An array of group definition objects.
+        joined_columns: An array of dict(s) that include an "alias" and "join_path" where,
+            "join_path" represents linkages via a simple many-to-many mapping to a column in another table.
         return_record_summaries: Whether to return self record summaries.
         table_record_summary_templates: A dict of record summary templates, per table.
     """
@@ -46,6 +49,7 @@ def list_records_from_table(
         _json_or_none(order),
         _json_or_none(filter),
         _json_or_none(group),
+        _json_or_none(joined_columns),
         return_record_summaries,
         _json_or_none(table_record_summary_templates),
     ).fetchone()[0]
@@ -56,6 +60,7 @@ def get_record_from_table(
     conn,
     record_id,
     table_oid,
+    joined_columns=None,
     return_record_summaries=False,
     table_record_summary_templates=None,
 ):
@@ -67,6 +72,8 @@ def get_record_from_table(
     Args:
         record_id: The primary key value of the record.
         table_id: The OID of the table whose record we'll get.
+        joined_columns: An array of dict(s) that include an "alias" and "join_path" where,
+            "join_path" represents linkages via a simple many-to-many mapping to a column in another table.
         return_record_summaries: Whether to return self record summaries.
         table_record_summary_templates: A dict of record summary templates, per table.
     """
@@ -75,6 +82,7 @@ def get_record_from_table(
         'get_record_from_table',
         table_oid,
         record_id,
+        _json_or_none(joined_columns),
         return_record_summaries,
         _json_or_none(table_record_summary_templates),
     ).fetchone()[0]
@@ -126,6 +134,7 @@ def list_by_record_summaries(
     offset=0,
     search=None,
     table_record_summary_templates=None,
+    linked_record_path=None,
 ):
     result = db_conn.exec_msar_func(
         conn,
@@ -135,6 +144,7 @@ def list_by_record_summaries(
         offset,
         search,
         _json_or_none(table_record_summary_templates),
+        _json_or_none(linked_record_path),
     ).fetchone()[0]
     return result