Avro Deserialization issue

[rohan-kro]

Hi team, i am trying to consume events from the desp. I am getting error in avro deserialization and this error is crashing the app.

Error message: avro : "invalid default for field timezone. is not a string"

I am able to consume the events using an older version v1.8.0 but when i try to use the newer versions i am unable to do so.

Can u help me here?

[nrwiersma]

Can you give a minimal example showing the problem?

[rohan-kro]

Check this schema:

Schema- {"type":"record","name":"ClassSpecificUpdated","namespace":"com.kroger.desp.events.itemcatalog.product","fields":[{"name":"eventHeader","type":{"type":"record","name":"EventHeader","namespace":"com.kroger.desp.commons.itemcatalog.product","fields":[{"name":"id","type":{"type":"string","avro.java.string":"String"},"doc":"A unique identifier of the event - for example, a randomly generated GUID"},{"name":"time","type":"long","doc":"Time the event occurred in milliseconds since epoch, UTC timezone."},{"name":"eventType","type":{"type":"string","avro.java.string":"String"},"doc":"Type of occurrence which has happened. Reference the domain.event registered in schema-registry."},{"name":"source","type":{"type":"string","avro.java.string":"String"},"doc":"Service that produced the event. Future: reference to producer registry."}]},"doc":"header field for this event"},{"name":"gtin","type":{"type":"string","avro.java.string":"String"},"doc":"The Global Trade Item Number is the standard 14-digit number used to identify all trade items in Item Management. GTIN for Item at the packaging level."},{"name":"alcoholProof","type":["null","double"],"doc":"Identifies the alcohol proof of the product.","default":null},{"name":"alcoholicBeverageContentType","type":["null",{"type":"record","name":"CodeNamePair","namespace":"com.kroger.desp.commons.itemcatalog.product","fields":[{"name":"code","type":{"type":"string","avro.java.string":"String"},"doc":""},{"name":"name","type":{"type":"string","avro.java.string":"String"},"doc":""}]}],"doc":"The descriptive term that is used by the manufacturer to identify a particular type of alcoholic beverage contained in a product.","default":null},{"name":"alcoholicCordialSyrupType","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"The descriptive term that is used by the product manufacturer to identify the name of the principal product on which the cordial/syrup is based.","default":null},{"name":"beerType","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"The descriptive term that is used by the product manufacturer to identify a particular type or variety of beer.","default":null},{"name":"spiritType","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"The descriptive term that is used by the product manufacturer to identify a particular type or variety of spirit.","default":null},{"name":"californiaRedemptionValueFlag","type":["null","boolean"],"doc":"Indicator the item requires a California Redemption Value","default":null},{"name":"californiaRedemptionValueAmount","type":["null","double"],"doc":"Dollar value of the California Redemption Value (CRV)","default":null},{"name":"nationalDrugCode","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"The 11 digit National Drug Code number for the consumer item.","default":null},{"name":"pseudoephedrineAmount","type":["null","double"],"doc":"Total milligrams of PSE in the package (NOTE: This is the total package amount not by dosage).","default":null},{"name":"sizeDescriptionOther","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the other size of the product.","default":null},{"name":"rating","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identifies the appropriate age audience.","default":null},{"name":"releaseDate","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identifies the date that the media, movie or entertainment was released to the market.","default":null},{"name":"sizeDescription","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"A descriptive term used to identify the size of the product.","default":null},{"name":"supplierStyleModelNumber","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Style or model number assigned by the product supplier mainly for general merchandise items e.g. Television","default":null},{"name":"title","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"The name of a book, composition, or other artistic work.","default":null},{"name":"msaiReporting","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the promotional description/codes of the product.","default":null},{"name":"requiresMsaiReportingFlag","type":["null","boolean"],"doc":"Identify if the product requires MSAi Reporting.","default":null},{"name":"numberOfPiecesInCustomerPackageFixed","type":["null","int"],"doc":"Identifies the (fixed) number of pieces in a consumer package.","default":null},{"name":"numberOfPiecesInCustomerPackageVariable","type":["null",{"type":"array","items":"int"}],"doc":"No. of Pieces (variable range) in consumer package","default":null},{"name":"numberOfPiecesInCustomerPackage","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identifies if the number of pieces in a consumer package is fixed or variable.","default":null},{"name":"bannerName","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Identify the Kroger Banner name of the card.","default":null},{"name":"cardIsClosedLoopFlag","type":["null","boolean"],"doc":"Identifies that the card can only be used at the merchant/merchants listed on the card.","default":null},{"name":"cardIsGeneralPurposeReloadableFlag","type":["null","boolean"],"doc":"Identifies if the card is a general purpose reloadable card.","default":null},{"name":"cardMayBeSubjectToApplicableTaxesFlag","type":["null","boolean"],"doc":"Identifies if the card is subject to applicable taxes.","default":null},{"name":"cardProcessor","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the processor of the card.","default":null},{"name":"cardReloadableFlag","type":["null","boolean"],"doc":"Identify if the card is reloadable.","default":null},{"name":"contactableCountriesRegions","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Identify the countries and regions that a card can be used to contact.","default":null},{"name":"containsExpirationDateFlag","type":["null","boolean"],"doc":"Identify if the product has an expiration date.","default":null},{"name":"coverageExceptions","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the exceptions to the coverage, national and international, not indicated by selection of states and/or countries/regions.","default":null},{"name":"dataAllowance","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the amount of data, in megabytes, permitted on a card per month.","default":null},{"name":"denominationOfCard","type":["null","double"],"doc":"Identify the dollar amount, with no decimals, of a card.","default":null},{"name":"expirationDateOfBonusCard","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the expiration date of the bonus card. Format MM/DD/YYYY","default":null},{"name":"feeAmountAssociatedWithCardPurchase","type":["null","double"],"doc":"Identifies the amount of the fee associated with the purchase of the card.","default":null},{"name":"feeAssociatedWithCardPurchaseFlag","type":["null","boolean"],"doc":"Identifies if there is a fee associated with the purchase of the card.","default":null},{"name":"feeEmbeddedInCardPriceFlag","type":["null","boolean"],"doc":"Identifies if the fee is embedded in the purchase price of the card.","default":null},{"name":"feePrintedOnCardFlag","type":["null","boolean"],"doc":"Identifies if the fee associated with the purchase of the card is printed on the card.","default":null},{"name":"forDomesticUseInEntireContinentalUnitedStatesFlag","type":["null","boolean"],"doc":"Identify if a card can be used domestically in the entire continental United States.","default":null},{"name":"lastDisplayDate","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the last date the card can be sold. Format MM/DD/YY","default":null},{"name":"minutesOnCard","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the number of minutes on a card.","default":null},{"name":"openLoopFlag","type":["null","boolean"],"doc":"Card can be used anywhere debit financial cards are accepted within the U.S.","default":null},{"name":"contactableCountriesRegionsOther","type":["null",{"type":"array","items":{"type":"string","avro.java.string":"String"}}],"doc":"Identify the other countries and regions that a card can be used to contact not on the list","default":null},{"name":"soldOnlyAtKrogerFuelCentersFlag","type":["null","boolean"],"doc":"Identify if the product is sold only at Kroger Fuel Centers.","default":null},{"name":"statesUnusableIn","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Identify the states that a card cannot be used in.","default":null},{"name":"taxEmbeddedInCardPriceFlag","type":["null","boolean"],"doc":"Identifies if the applicable tax is embedded in the purchase price of the card.","default":null},{"name":"telecomCarrier","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identify the telecom provider.","default":null},{"name":"textsAllowance","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the number of text messages permitted on a card per month.","default":null},{"name":"airtimeCardType","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identify the type of airtime card","default":null},{"name":"phoneCardType","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Identify the type of phone card.","default":null},{"name":"valueOfCard","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identifies if the denomination of the card is fixed or variable.","default":null},{"name":"variableDenominationOfCardMaximumAmount","type":["null","double"],"doc":"Identifies the maximum amount of the denomination of the card when variable.","default":null},{"name":"variableDenominationOfCardMinimumAmount","type":["null","double"],"doc":"Identifies the minimum amount of the denomination of the card when variable.","default":null},{"name":"bonelessClaim","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identifies that item is boneless","default":null},{"name":"harvestType","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identifies whether the fish / seafood product is wild caught or farm raised.","default":null},{"name":"inStorePreparedPackagedFlag","type":["null","boolean"],"doc":"Indicates the product was prepared or packaged in the store.","default":null},{"name":"intendedToBeUsedAsToppingOrIngredientFlag","type":["null","boolean"],"doc":"Indicates whether or not the product is intended to be used as a topping or ingredient","default":null},{"name":"itemBoneInFlag","type":["null","boolean"],"doc":"Determines if the meat contains bone.","default":null},{"name":"iTradeCompliantFlag","type":["null","boolean"],"doc":"Identify if the item is iTrade compliant","default":null},{"name":"recommendedShelfLifeOnceSealBrokenOnPackage","type":["null","double"],"doc":"Identify the recommended shelf life (in days) once the seal is broken on manufacurer packaging and the original expiration date is null and void.","default":null},{"name":"scannableBarcodeAffixedToConsumerPackageFlag","type":["null","boolean"],"doc":"Identify if the consumer unit has a scannable barcode affixed to the consumer package.","default":null},{"name":"shelfLifeOfProductWithoutExpirationDateOnPackage","type":["null","double"],"doc":"What is the recommended Shelf life (in days) for product that will not have scale label applied and/or does not have expiration date printed on package","default":null},{"name":"shelfLifeOfUnopenProductReceivedFrozenSoldThawed","type":["null","double"],"doc":"What is the recommended shelf life (in days) for unopened product that was received frozen then thawed and held in cooler for backstock and/or production","default":null},{"name":"targetRetailSellingSpace","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Indicates the area of the store that is intended for item to be sold.","default":null},{"name":"processingRequiredAtStoreLevelType","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Identify the type of processing the product required at store level.","default":null},{"name":"usdaMeatGrade","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identifies the USDA meat grade","default":null},{"name":"floralContainerType","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Type of container a floral product is in","default":null},{"name":"itemHasPmaPluFlag","type":["null","boolean"],"doc":"Identifies if the product contains a Produce Marketing Association PLU sticker.","default":null},{"name":"floralContainerTypeOther","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identifies the type of floral container a product is in that is not on the list.","default":null},{"name":"potPlanterSizeLiquidMeasure","type":["null","int"],"doc":"The descriptive term that is used by the product manufacturer to identify the volume of the pot or planter in fluid ounces.","default":null},{"name":"pottedPlantFlag","type":["null","boolean"],"doc":"Identifies whether or not the plant is potted.","default":null},{"name":"shippingMaterial","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identifies the type of shipping material a product is shipped with.","default":null},{"name":"numberOfServingsInConsumerUnit","type":["null",{"type":"array","items":"double"}],"doc":"Number of servings in any package (if variable range put minimum and maximum value)","default":null},{"name":"servingsPerPackage","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identify the measurement type of serving per package (i.e. Fixed or Range)","default":null},{"name":"milligramsOfNicotine","type":["null","double"],"doc":"Identify the milligrams of nicotine in the product.","default":null},{"name":"milliliterOfLiquidVolume","type":["null","double"],"doc":"Identify the milliliters of liquid volume.","default":null},{"name":"packsPerCarton","type":["null","int"],"doc":"Identifies the number of packs per carton.","default":null},{"name":"tobaccoType","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identifies a particular type or variety of tobacco.","default":null},{"name":"unitPerPack","type":["null","int"],"doc":"Identifies the number of units per pack.","default":null},{"name":"ageRestricted","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identify the age group that product is restricted from sale.","default":null},{"name":"cheeseProductType","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Identifies the type of cheese.","default":null},{"name":"consumerLifeStage","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Identifies the period or stage in the consumer's life during which the product is considered to be suitable.","default":null},{"name":"consumerUnitResealableFlag","type":["null","boolean"],"doc":"Identify if Consumer Unit is Resealable","default":null},{"name":"containsFlourFlag","type":["null","boolean"],"doc":"If product contains flour, it is considered \"food\" in some states and therefore taxed at a lower rate","default":null},{"name":"guaranteedAnalysis","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Percentages of crude fats, fibers, proteins, maximum moisture and other nutrients.","default":null},{"name":"itemIntendedForHumanConsumptionFlag","type":["null","boolean"],"doc":"Items packaged / designated for human consumption","default":null},{"name":"juicePercentage","type":["null","double"],"doc":"Percentage of juice in the product (i.e., 10 or 50 or 100). ","default":null},{"name":"labelingIncludes","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Indicates whether the labeling from the vendor includes ingredients, UPC/GTIN, other.","default":null},{"name":"levelOfCooking","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"The extent to which heat is used to preserve product quality and maintain wholesomeness.","default":null},{"name":"levelOfSugarSucroseSweetenerClaim","type":["null","com.kroger.desp.commons.itemcatalog.product.CodeNamePair"],"doc":"Amount of sugar or sweetener contained in product, for instance, low sugar, no sugar","default":null},{"name":"methodOfSweetening","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Attribute describes method of sweetening of product. Product may be packed in natural juices, heavy syrup with sugar, or artificially sweetened with Splenda","default":null},{"name":"milkProductSubstituteFlag","type":["null","boolean"],"doc":"Identifies if the product is a milk or milk substitute.","default":null},{"name":"netContentEquivalentUomOverrideFlag","type":["null","boolean"],"doc":"Use this field only if the net content of the item being submitted is not able to be quantified using the UOMs provided in Required Net Content EUOM.","default":null},{"name":"colorOther","type":["null",{"type":"array","items":{"type":"string","avro.java.string":"String"}}],"doc":"Indicates additional potential colors not included on the valid value list for attribute 588 Color.","default":null},{"name":"productIsLabeledWithAnFdaApprovalNationalDrugCodeFlag","type":["null","boolean"],"doc":"Identifies if the product has a valid National Drug Code (NDC).","default":null},{"name":"productionState","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Indicates the state in which the product was produced.","default":null},{"name":"usdaWicEligibilityForVendorFlag","type":["null","boolean"],"doc":"Designation by Vendor if item is currently USDA WIC eligible. Informational use.","default":null},{"name":"wholeGrainPercent","type":["null","double"],"doc":"Percentage of whole grains: percentage of the product containing all the essential and naturally-occurring nutrients of the whole grain seed","default":null},{"name":"wholeGrainsContainedInProductFlag","type":["null","boolean"],"doc":"Product is manufactured using whole grains.","default":null},{"name":"artist","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identifies the name of the artist or artists of a piece of music or entertainment.","default":null},{"name":"assortmentPlanName","type":["null",{"type":"string","avro.java.string":"String"}],"doc":"Identify the Assortment Plan Name","default":null},{"name":"genre","type":["null",{"type":"array","items":"com.kroger.desp.commons.itemcatalog.product.CodeNamePair"}],"doc":"Identifies the category characterized by similarities in form, style, or subject matter.","default":null},{"name":"itemSentTimestamp","type":["null",{"type":"record","name":"DateTime","namespace":"com.kroger.desp.commons.itemcatalog.product","fields":[{"name":"value","type":{"type":"string","avro.java.string":"String"},"doc":""},{"name":"timezone","type":{"type":"string","avro.java.string":"String"},"doc":"","default":null}]}],"doc":"Local timestamp added upon item send to Kafka topic. The value is UTC 0, or UTC Z, also known as Greenwich Mean Time with no offset, and the timezone is always set to local, which is UTC-5 with daylight savings, or America/New_York, otherwise known as Eastern Standard Time.","default":null}]}

This is the attribute schema for which we are seeing this error.

{"name":"timezone","type":{"type":"string","avro.java.string":"String"},"doc":"","default":null}]}],

For this "timezone" attribute we are seeing this error. When its value is not present its defaulted to null.
Also we are seeing this error in the newer versions . Its working as expected in v1.8.0

[nrwiersma]

null is not a valid default for a string type.

[rohan-kro]

Even though null is not a valid default for a string type , we are able to deserialize the above schema in older versions that is v1.8.0 but when I upgrade it to v2 version i am getting an error.

[nrwiersma]

v2 does better error checking on the schema when parsing it.