Conversation
Test Results5 091 tests - 49 5 087 ✅ - 49 2m 22s ⏱️ +9s Results for commit 4992f43. ± Comparison against base commit 2a02a86. This pull request removes 5140 and adds 5091 tests. Note that renamed tests count towards both.This pull request removes 4 skipped tests and adds 4 skipped tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
|
Hey @arlimus is this feature list the final one or will in the two weeks from 02.02-16.02 more idears/changes be collected ? Saw it by accident and am just curious |
The doors are open!! We can collect any other breaking features until the 16th :) Of course we can also add more features after and make them afterwards, it's really just the crazy breaking changes that we can finally make in these 2 weeks |
|
I added the companion v13 for cnspec here and to the description above: mondoohq/cnspec#2057 |
|
I can only point to this where fuzzy search is still missing Best Regards! |
I got you covered! I flagged it above as "optional chaining of resources" as a todo item for v13 :) We will run down all the policies next week because we need to update/measure the impact of this change. It's tough to do it all in one quick change and keep things stable, but so far I'm optimistic we can get it tackled. I'll share the spec next week, so far I'm thinking optional chaining as seen in ts to make it explicit when users want chaining and error otherwise (as you would in js/ts) |
|
Let me explicitly call out fuzzy above |
|
I know that we will have some allignement calls soon. Without naming this feature requests as they will go over kai. I am just asking as our team is curious if these things are already there or planned. So dont take this as a feature request yet :) Is there a yaml schema to validate syntax/structure? Is there a possibility to share/inherit props (variables) between "policies" and global "queries" (planned)? Which possibilities of automated tests does Mondoo offer? There is a mock provider, but no documentation for it |
Yeah, I think if we add it it is something we can just do on top, since it wouldn't be breaking change. let me track it as a stretch idea, if we hit the v13 great and if not we just do it after |
|
|
We are in the beginning phase of writing custom policies so i cant give the full picture yet. This needs some more weeks. We would try to be as modular as we can be and merge the Policy either on CI or reference on the bigger policy file from other smaller policy files (checks or queries). One reason why we want to be modular is because of git. To avoid MR conflicts when more people work on the same file. We planned to write and try to write policies similar to what you already shown on your policies queries can be written under But we find it much more readable to see the whole structure at top under policies and put all the query logic under The So we said that we define all queries on purpose on the global level part. Which you also did in your policies for gcp and aws atleast. For example we have a query which checks for specific tags This check we will not only use for s3 but for all of them under us + to be continued: "AWS::ApiGateway::RestApi",
"AWS::DynamoDB::Table",
"AWS::EC2::Instance",
"AWS::EC2::SecurityGroup",
"AWS::ElasticLoadBalancingV2::LoadBalancer",
"AWS::EFS::FileSystem",
"AWS::RDS::DBInstance",
"AWS::S3::Bucket",
"AWS::Elasticsearch::Domain",
"AWS::OpenSearch::Domain",To avoid hardcoded values and to enable parameterize logic for different customers we want to define these props once per policy file for the Currently we either have to hardcode the value, define the props in every 11 checks for just this one requirement or use props under Summary:
|
|
We are missing this feature described in the documentation bundle-global-props-deprecated error test.mql.yaml 168 Defining global properties in a policy bundle is deprecated. Define properties within individual policies and queries instead. So we did not plan with it as we thought it will be potentially removed soon.
The other way around we can define props on root level but they are not inherited on quecks under policies. Hope this sums it up |
Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
This introduces optional chaining in MQL. It also changes the behavior so that by default we do not blindly chain elements anymore but instead print errors when an element in the chain is `null`. **Background** Since MQL and cnspec are often used for security and compliance policies, we have to create a careful balance between user-comfort and precision. This means that e.g. a comparison like `user.id == 123` will work perfectly well, even if we compare strings and numbers, because the users intention is understood. However, when chaining calls, we have so far sided on the side of comfort - maybe a bit too much. The current behavior - which is optional chaining by default - can lead to cases where the user intent is misunderstood. It's ok if an AI hallucinates (not really, but you get the point), but it's not ok for a query/policy engine to lead to such negative unexpected behaviors. You can see a full example on display here: #6428 **Action** The system makes all chaining mandatory non-null by default. This means that if you have json like this: ```json { "a": { "b": "c" } } ``` and called it with: ``` > parse.json('example.json').params.d.b ``` it would now produce an error: ``` > parse.json('example.json').params.d.b == "c" [failed] parse.json.params.d.b == "c" error: cannot access field "b", parent element is null ``` This errors is new. It's becase the field `d` doesn't exist in this JSON. This is where **optional chaining** comes back into the picture: It allows you to say "please don't print an error, just see if you can get the value" and thus use the previous behavior of MQL: ``` > parse.json('example.json').params.d?.b == "c" [failed] parse.json.params.d.[]? == "c" expected: == "c" actual: _ ``` This is also in line with behaviors in JavaScript and TypeScript.
tldr: Introduce `having` on lists/maps, which acts like a `where` filter that also checks that the resulting list is not empty. ```coffee my.collection.having( some condition ).all( ... ) ``` **Problem** As explained by @LittleSalkin1806 in #6633 (comment) > If the bucket has no tags at all, the where clause will return an empty list, the values will take values from no existing key (empty list) and the all function will evaluate to true because there are no elements that violate the condition. This can lead to false positives in the compliance checks. I ran into this same problem way too often myself! I've been working on a function to address this problem. The `where` clause behaves like a traiditional `filter` in JS/TS and is close to SQL, so we wanted to keep their behavior/meaning. However, there is clearly a need for the combination, because this pattern happens way too often: ```coffee collection.where( condition ) != empty collection.where( condition ).otherCalls... ``` Using @LittleSalkin1806 's example: ```coffee aws.s3.bucket.tags.where(key.downcase == 'dtit:sec:infosecclass') != empty aws.s3.bucket.tags.where(key.downcase == 'dtit:sec:infosecclass').values.map(downcase).in([...]) ``` **Solution** This PR introduces the `having` keyword. It acts like `where` that also makes sure that the list is not empty: ```coffee collection.having( condition ).otherCalls... ``` or with the above example: ```coffee aws.s3.bucket.tags.having(key.downcase == 'dtit:sec:infosecclass').values.map(downcase).in([ ... ]) ``` If the filtered list is empty, the query fails. If it is non-empty, it uses the filtered values and looks at the next condition. Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
- Add prerelease: true to goreleaser-unstable config - Print github.ref, github.ref_name, and github.event_name for debugging Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
We plan to move to new IDs in v14, but we need this release to make it possible, since cross-provider calls are codified from now on. Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
4 participants
Welcome to cnquery v13 / cnspec v13
Note: I'm writing out the description as things evolve, please ask questions below.
Timeline
User-facing changes
unify the IDs of providers and clean up the version-handlingmoving the new IDs to v14 for compatibility reasonsAutoUpdateEnginein v13 ✨ gate auto-updates behind feature-flag in v13 #6529👋 cnquery, 🤗 mql
What's going on: We have a ton of confusion between cnquery and cnspec, explaining both CLIs over and over again.
Let's simplify things: Cnquery will be renamed to MQL, since it's at the core of the querying engine anyway. The providers and all their data are tighly connected to MQL as well. The only element not part of MQL is the execution of querypacks.
We will move querypacks over to cnspec, which already contains policies (and frameworks) anyway. By moving it out, this entire project focuses on data-collection and querying only.
Internal changes
Post v13 cleanup
Stretch goals
breaking (pre-release only)
Is400AccessDeniedErrorneed proper error handling, otherwise results appear incorrectnon-breaking (at any point)
_.prefix, see https://github.com/mondoohq/server/pull/14241x { field }andx.fieldbeing the resource) then it cannot be private (it has to be public)Other changes that happened in latest v12, that are now streamlined in v13