Can you refer any documentation that explains how to use MapKeys, MAP and MapValues functions?

[schinchanikar-rms]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce

Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

If you think the bug depends on external factors (e.g., dataset), please provide us with a minimal reproducible example that consists of the following items:

• a minimal dataset, necessary to reproduce the error
• the minimal runnable code necessary to reproduce the error, which can be run on the given dataset
• the necessary information on any used packages, .NET runtime version, and system it is run on
• in the case of random processes, a seed for reproducibility

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

[imback82]

Does the following help? https://sparkbyexamples.com/spark/spark-sql-map-functions/

[dannydwarren]

I'm not seeing .map() on DataFrame is this question about the lack of documentation to find and use .map() in C#?

[imback82]

The original question is on map functions defined in Microsoft.Spark.Sql.Functions.

There is no Map() that operates on DataFrame. However, you can achieve the same using Select() along with UDF. Please let me know if you have a specific scenario you are trying to achieve.

[dannydwarren]

Forgive me if this convo should be somewhere else...

We are using Confluent.Kafka.Avro and need to deserialize a column in a DF to a GenericRecord type. When I tried using a UDF I ran into an error when returning a GenericRecord because it does not translate to a Sql Type. We were using the RegisterUdf api and doing a SelectExpr to call that registered UDF. Is there a way to return arbitrary objects from a UDF?

[imback82]

We are working a feature where you can return an arbitrary Row object in #117. Would that cover your scenario?