[Feature][HTTP] Set time parse format at HTTP Source

[ruanwenjun]

Search before asking

• I had searched in the feature and found no similar feature requirement.

Description

When I use the HTTP source to read data from an endpoint.

I got an error because a time field parse failed.

Caused by: java.time.format.DateTimeParseExceptionreate breakpoint : Text 2023-08-18 10:52:19'could not be parsed at index 10

I find we don't support to set the time format at HTTP source config, so when we have a specific time string, we can only use string as the field type.

Furthermore, I find in some other connector, we support to config a time format which will be used to parse the time field.

But this cannot work when we have different time formats in one schema.

Can we set this under the field level?

schema {
    field {
        id {
            type = int
            xx
        }
        createTime {
            type = timestamp
            format_type = "yyyyMMdd HH:mm:ss"
        }
   }


}

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[wu-a-ge]

In my opinion, connectors that use text format and JSON format should be able to customize the format of date-time, date, and time data

[ruanwenjun]

In my opinion, connectors that use text format and JSON format should be able to customize the format of date-time, date, and time data

If we define one field is timestamp, but the format string is not equals with the JSON format, e.g. yyyyMMdd HH:hh:mm is a useful format string, but default JSON cannot deserialize this.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[FuYouJ]

could we add support for multiple formats to formater to try parsing time formats?

[ruanwenjun]

could we add support for multiple formats to formater to try parsing time formats?

If we can expose the formatter to conf it will be great, use multiple formats is not good to me, we cannot provide all format.

[liugddx]

could we add support for multiple formats to formater to try parsing time formats?

If we can expose the formatter to conf it will be great, use multiple formats is not good to me, we cannot provide all format.

Does ScheamTable currently have parameters that can be stored in format? I haven't seen it yet