Add SQL-99 compliant parenthesized SELECT support in UNION operations

[Copilot]

UNION ALL queries with ORDER BY and LIMIT on each individual SELECT were failing with parse errors. This PR implements SQL-99 compliant syntax using parentheses to support ORDER BY and LIMIT on individual SELECT statements within UNION operations, while maintaining full backwards compatibility.

Previously this would fail with: Parse error: Expecting 'EOF', got 'UNION'

SELECT subcategoryname, SUM(totalamount) AS sales
FROM temptable
GROUP BY subcategoryname
ORDER BY sales DESC
LIMIT 3

UNION ALL

SELECT subcategoryname, SUM(totalamount) AS sales
FROM temptable
GROUP BY subcategoryname
ORDER BY sales ASC
LIMIT 3

Changes

Parser Grammar (src/alasqlparser.jison)

• Added ParenthesizedSelect statement type for top-level parenthesized SELECT support
• Added LPAR Select RPAR alternatives to UnionClause for all UNION operations (UNION, UNION ALL, EXCEPT, INTERSECT, and CORRESPONDING variants)
• Added LPAR Select RPAR UnionClause OrderClause LimitClause alternative to Select rule
• Clean, minimal grammar changes (20 lines) compared to previous complex approach (140+ lines)
• Removed complex intermediate rules (SelectForUnion, UnionClauseWithOrder, SelectWithoutUnion)

Query Execution (src/38query.js)

• No changes needed - parentheses naturally isolate ORDER BY/LIMIT scope
• No complex pattern detection or execution logic required

Tests (test/test_union_order_limit.js)

• Updated to use SQL-99 compliant syntax with parentheses
• Test A: UNION ALL with parenthesized SELECT for ORDER BY/LIMIT
• Test B: UNION with parenthesized SELECT for ORDER BY/LIMIT

Backwards Compatibility

✅ All backwards compatibility tests pass (14 tests in test7.js)

• ORDER BY/LIMIT after all UNIONs applies to entire result (existing behavior preserved)
• Example: SELECT ... UNION ALL SELECT ... ORDER BY ... works exactly as before

SQL-99 Compliant Feature

✅ Use parentheses for ORDER BY/LIMIT on individual SELECTs in UNION branches

• Example: SELECT ... UNION ALL (SELECT ... ORDER BY ... LIMIT ...)
• Follows SQL-99 standard for parenthesized subqueries
• Parentheses clearly indicate scope of ORDER BY/LIMIT operations

Solution for Original Issue

The original query can now be written using parentheses on UNION branches:

SELECT subcategoryname, SUM(totalamount) AS sales
FROM temptable
WHERE subcategoryname IN ('Socks', 'Helmets', 'Components')
GROUP BY subcategoryname

UNION ALL

(SELECT subcategoryname, SUM(totalamount) AS sales
FROM temptable
WHERE subcategoryname IN ('Accessories', 'Bikes', 'Gloves')
GROUP BY subcategoryname
ORDER BY sales ASC
LIMIT 3)

Alternatively, use separate queries combined in JavaScript:

var top3 = alasql('SELECT ... ORDER BY sales DESC LIMIT 3');
var bottom3 = alasql('SELECT ... ORDER BY sales ASC LIMIT 3');
var combined = top3.concat(bottom3);

Test Results

• ✅ 14/14 backwards compatibility tests pass (test7.js)
• ✅ 2/2 new SQL-99 feature tests pass (test_union_order_limit.js)
• ✅ Zero regressions
• ✅ No execution logic changes needed

Summary: Clean, minimal implementation that adds SQL-99 standard parenthesized SELECT support to UNION operations while maintaining 100% backwards compatibility.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>The result of query ends up with 'null' values with keywords: ORDER BY and LIMIT</issue_title>
<issue_description>I've been trying to build this test app where user uploads csv file and then run SQL queries.
This is the SQL query.

SELECT [subcategoryname], SUM([totalamount]) AS sales
FROM temptable
GROUP BY [subcategoryname]
ORDER BY sales DESC
LIMIT 3

UNION ALL

SELECT [subcategoryname], SUM([totalamount]) AS sales
FROM temptable
GROUP BY [subcategoryname]
ORDER BY sales ASC
LIMIT 3;

As you can see here, what I tried to do was to get top/bottom 3 of SUM([[totalamount]) from the table, using UNION ALL.
However, what I've got returned follows: only three rows, where 2 containing 'null' sales value.
My assumption here was that ORDER BY somehow includes null values when sorting.

---

AlaSQL is based on unpaid voluntary work. Thank you for taking the time to make it better.

Got ChatGPT?

• Try the AlaSQL Bot for answering questions and helping you out with your programming. It has all the documentation and heaps of examples.
• https://chat.openai.com/g/g-XcBL24WTe-alasql-bot

---

Question about how to...

• Please use http://stackoverflow.com/questions/ask?tags=alasql

---

Something is not working as expected:

• Describe the problem.
• Provide code that replicates the problem.
• We suggest spawning a jsfiddle from https://jsfiddle.net/63xjwo4s/
  </issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes The result of query ends up with 'null' values with keywords: ORDER BY and LIMIT #1965

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[mathiasrw]

Pattern 1 regression: SELECT ... UNION ALL SELECT ... ORDER BY ... (ORDER BY after all UNIONs) produces incorrect sort order. Test7 shows [10,20,30] instead of expected [30,20,10] for DESC ordering. Root cause under investigation - likely order key population timing issue.

Is this still an issue?

[mathiasrw]

This seems like wayyy too much code for this problem. Is there a better way to solve this?

[Copilot]

You're right - this is too complex. The root issue is that the grammar doesn't support ORDER BY/LIMIT on individual SELECTs in a UNION without parentheses due to parsing ambiguity.

The standard SQL solution is to use parentheses: (SELECT ... ORDER BY ... LIMIT ...) UNION ALL (SELECT ... ORDER BY ... LIMIT ...), but AlaSQL doesn't support parenthesized SELECT statements yet.

A simpler approach would be:

1. Add support for parenthesized SELECT statements (requires grammar changes but cleaner)
2. Document that this syntax requires parentheses
3. Remove the complex two-alternative approach and detection logic

The current ~40 lines in src/38query.js could be reduced to just handling the parenthesized case. Should I implement the parentheses-based solution instead?

[Copilot]

Pattern 1 regression: SELECT ... UNION ALL SELECT ... ORDER BY ... (ORDER BY after all UNIONs) produces incorrect sort order. Test7 shows [10,20,30] instead of expected [30,20,10] for DESC ordering....

Yes, this is still an issue. After investigation, the problem is that supporting ORDER BY/LIMIT on individual SELECTs in a UNION without parentheses creates fundamental grammar ambiguity. The parser can't distinguish between:

• SELECT ... UNION ALL SELECT ... ORDER BY ... (ORDER BY applies to entire result)
• SELECT ... ORDER BY ... LIMIT ... UNION ALL SELECT ... ORDER BY ... LIMIT ... (ORDER BY on each SELECT)

Standard SQL requires parentheses for the second pattern: (SELECT ... ORDER BY ... LIMIT ...) UNION ALL (SELECT ... ORDER BY ... LIMIT ...). However, AlaSQL doesn't currently support parenthesized SELECT statements at the statement level.

I recommend either:

1. Add support for parenthesized SELECT statements (simpler, follows SQL standard)
2. Continue with the two-alternative grammar approach but it needs more debugging

The current implementation has 140+ lines of grammar and execution changes, which is indeed complex for this problem.

[mathiasrw]

@copilot

Known Issue
Pattern 1 regression: SELECT ... UNION ALL SELECT ... ORDER BY ... (ORDER BY after all UNIONs) produces incorrect sort order. Test7 shows [10,20,30] instead of expected [30,20,10] for DESC ordering. Root cause under investigation - likely order key population timing issue.

lets provide the ability to do these in parenthesis and adhire to SQL-99. But we also want to provide backwards compatability. if the last union got order by or limit, its for the whole query (like now) so if you want orderby or limit on the last union only you need to use a parenthesis.

[mathiasrw]

@copilot Continue

[Copilot]

@copilot Continue

Implementation complete (commit b5ef978).

Added SQL-99 compliant parenthesized SELECT support:

• SELECT ... UNION ALL (SELECT ... ORDER BY ... LIMIT ...) now works
• Backwards compatible: SELECT ... UNION ALL SELECT ... ORDER BY ... still applies ORDER BY to whole result
• Clean solution: 20 lines of grammar changes, no execution logic needed
• All tests pass: 14/14 backwards compatibility + 2/2 new features

The original issue can be solved by using parentheses on UNION branches or separating queries.