Personalization causes duplicate `to` email and sometimes causes multiple emails

[fangbyte]

Issue Summary

When including a BCC array, the sendgrid-ruby client either errors if an additional to is not provided, and when an additional to is provided can intermittently cause multiple duplicate emails to be sent.

Steps to Reproduce the Issue

Following the kitchen sink example from https://github.com/sendgrid/sendgrid-ruby/blob/6.3.3/examples/helpers/mail/example.rb#L21

  require 'sendgrid-ruby'
  include SendGrid

  from = Email.new(email: 'from@example.com', name: 'Personalization Test')
  to = Email.new(email: 'to@example.com')
  subject = 'personalization test'
  content = Content.new(type: 'text/html', value: '<p>Hello</p>')

  mail = SendGrid::Mail.new(from, subject, to, content)

  # Add the BCC email 
  personalization = Personalization.new
  personalization.add_bcc(Email.new(email: 'bcc@example.com'))

  mail.add_personalization(personalization)
  mail.to_json

The output of mail.to_json (not actually JSON, but a ruby Hash)

{
  "from"=>{
    "email"=>"from@example.com",
    "name"=>"Personalization Test"
  },
 "subject"=>"personalization test",
 "personalizations"=>[ 
    {
      "to"=>[
        {
          "email"=>"to@example.com"
        }
      ]
    },
    {
      "bcc"=>[
        {
          "email"=>"bcc@example.com"
        }
      ]
    }
  ],
 "content"=>[
    {
     "type"=>"text/html",
     "value"=>"<p>Hello</p>"
    }
  ]
}

Note that a to array is present in the only entry in the personalizations array.

However if you try to send this email, you get the following error

resp = sendgrid_api.client.mail._('send').post(request_body: mail.to_json)
=> #<SendGrid::Response:0x00007fc3962ca4a0 @status_code="400", @body="{\"errors\":[{\"message\":\"The to array is required for all personalization objects, and must have at least one email object with a valid email address.\",\"field\":\"personalizations.1.to\",\"help\":\"http://sendgrid.com/docs/API_Reference/Web_API_v3/Mail/errors.html#message.personalizations.to\"}]}", @headers={"server"=>["nginx"], "date"=>["Thu, 06 May 2021 21:27:40 GMT"], "content-type"=>["application/json"], "content-length"=>["288"], "connection"=>["close"], "access-control-allow-origin"=>["https://sendgrid.api-docs.io"], "access-control-allow-methods"=>["POST"], "access-control-allow-headers"=>["Authorization, Content-Type, On-behalf-of, x-sg-elas-acl"], "access-control-max-age"=>["600"], "x-no-cors-reason"=>["https://sendgrid.com/docs/Classroom/Basics/API/cors.html"], "strict-transport-security"=>["max-age=600; includeSubDomains"]}>

To work around this issue we began explicitly adding the to email to the Personalization object before adding the BCC like so

 # Code from previous example

 personalization = Personalization.new

# This matches the usage in https://github.com/sendgrid/sendgrid-ruby/blob/6.3.3/examples/helpers/mail/example.rb#L21
 personalization.add_to(to)
 
 personalization.add_bcc(Email.new(email: 'bcc@example.com'))

 mail.add_personalization(personalization)

 mail.to_json

And the output of mail.to_json

{
  "from"=>{
    "email"=>"from@example.com",
    "name"=>"Personalization Test"
  },
 "subject"=>"personalization test",
 "personalizations"=>[
    {
      "to"=>[
        {
          "email"=>"to@example.com"
        }
      ]
    },
   {
      "to"=>[
        {
          "email"=>"to@example.com"
        }
      ],
      "bcc"=>[
        {
          "email"=>"bcc@example.com"
        }
      ]
    }
  ],
  "content"=>[
    {
      "type"=>"text/html",
      "value"=>"<p>Hello</p>"
    }
  ]
}

Notice there are now TWO entries in the Personalization array, and each one specifies a to email.

This request does succeed

resp = sendgrid_api.client.mail._('send').post(request_body: mail.to_json)
=> #<SendGrid::Response:0x00007fc3a621a540 @status_code="202", @body="", @headers={"server"=>["nginx"], "date"=>["Thu, 06 May 2021 21:36:32 GMT"], "content-length"=>["0"], "connection"=>["close"], "x-message-id"=>["_qw3BAbFQIiHzJbf7_oXBA"], "access-control-allow-origin"=>["https://sendgrid.api-docs.io"], "access-control-allow-methods"=>["POST"], "access-control-allow-headers"=>["Authorization, Content-Type, On-behalf-of, x-sg-elas-acl"], "access-control-max-age"=>["600"], "x-no-cors-reason"=>["https://sendgrid.com/docs/Classroom/Basics/API/cors.html"], "strict-transport-security"=>["max-age=600; includeSubDomains"]}>

The Issue

We opened a support case with the SendGrid team. We were told in no uncertain terms by a support rep named Anthony that the duplicate to fields were causing multiple copies of the email to be sent from one API request.

However, in our experience we only see the issue with duplicate emails intermittently, even for identical request bodies.

When we asked about the intermittent nature of our experience we were told that support does not review code and the same "resolution" was repeated in different words blaming our code generation for the issue.

If the multiple to fields are truly the issue, it's a serious bug in the SendGrid client library that the first code example that generates a single to array in the Personalizations object is not accepted.

Technical details:

• sendgrid-ruby version: 6.3.3
• ruby version: 2.5

[shwetha-manvinkurke]

This issue has been added to our internal backlog to be prioritized. Pull requests and +1s on the issue summary will help it move up the backlog.