Bug: Can't buy puerto rican numbers with phone number buying

[ibrand]

Describe the bug
Currently phone number buying is only guaranteed to work for US numbers. I was specifically trying to purchase Puerto Rican numbers when I saw a problem emerge.

To Reproduce
Steps to reproduce the behavior:

1. Go to the phone numbers tab
2. Click on the plus button to buy numbers and type in a non-us area code
3. See the spinner indicating an attempt to buy but a failure

Expected behavior
We should allow the purchasing of all types of phone numbers.

Additional context
A few notes here:

• Switchboard is also only able to buy US numbers
• Some of the numbers I eventually bought through switchboard hacking were only able to text domestically. Maybe worth eventually adding a toggle for purchasing numbers that can only text internationally?
• Here is the code for searching for numbers. One note here is that when I spoke to a Twilio rep, they said that for puerto rican numbers, you can't use the areaCode key as part of the availablePhoneNumber resource. Instead I ended up needing to use the contains like this:

  return client
    .availablePhoneNumbers('PR')
    .local.list({
      contains: '<the-actual-digits-of-the-area-code>*******',
      smsEnabled: true,
    })

It'd be nice to find a way to write this code so that it is maximally versatile and not full of logic forks.

[ibrand]

@hiemanshu have you tried out the phone number buying UI? Have you seen problems trying to buy numbers for other countries?

[maxkc]

@ibrand AvailablePhoneNumberLocal assumes you're buying in the United States unless told otherwise. Even though Puerto Rico numbers are +1, ISO 3166 alpha-2 assigns it PR.

Using contains: +1787 would work but then you'd have to look again for +1939 if 787 isn't available. It'll probably be easier to just rely on the country code instead.

This should work:

client.availablePhoneNumbers('PR')
      .local
      .list({smsEnabled: true})

It might be worth reminding users in-app that PR numbers cost more than US and don't support MMS. Because this is "domestic" traffic based off the country calling code of +1 sending an SMS from US --> PR won't error out but it'll cost more (and sending an MMS will error out).

[maxkc]

FWIW Twilio should buy 787 before buying 939 but PR is a NANP distributed overlay so users shouldn't have a preference between the two.

EDIT:

Maybe worth eventually adding a toggle for purchasing numbers that can only text internationally?

You can, generally, make most numbers text wherever you want on Twilio's end if you try hard enough. It'd probably be easier/better long term to enforce this in your code instead.

I think the hacky way would be to check that the country calling code of both numbers match prior to sending, this doesn't help with PR though because both would be +1. You could also incorporate MaxPrice so you'll get a 30010 error if something is too expensive (likely due to it being international) or use Lookup to validate that the countryCode or MobileCountryCode is as expected prior to loading the contact record into Spoke - both of these approaches would flag PR because it's more expensive and has a different countryCode.

[ibrand]

Oh this is interesting. It's a really good note to mark that if users are trying to buy puerto rican numbers that it'll cost more. Would our current code pose problems for other countries? I haven't tried buying numbers from other countries yet, and I think we should make sure to support that.

Sounds like the right implementation is to fork the logic if the area codes requested are 787 or 939 for special puerto rico cases.

[maxkc]

Would our current code pose problems for other countries?

Hm I'm not sure. I bet Matteo would know or I could poke around this weekend and see. I'd expect that code "supports" international in that it could buy international numbers, but the number formatting would cause an error somewhere along the line. +1 numbers (US, Canada, etc) should be fine as long as you change the ISO country code like I did above, but a Bristish number will be 12 digits instead of 11, which will probably break something along the line. Looks like GetUp made some edits to support AU numbers at one point.

Supporting MaxPrice as an admin setting is probably the simplest thing to do in the medium-short term (but also it shouldn't be that pressing because these are edge cases).

The PhoneNumber Resource beta API kinda solves some of this by relying on phoneNumberSid as a standard instead of the phone number itself, eliminating most of the need to worry about how many digits a number contains, but most of that isn't backwards compatible/might not be ever.

Buying/not buying certain countries can be enforced in the platform code or by the user in Twilio Console adding to confusion too. You can prevent purchasing numbers using HighRiskSpecialPrefix (and other associated resources) but a user can also do this with the Regulatory Bundles feature, which would override code... so "expected behavior" could vary widely depending on settings. This would be most problematic if the product was being used somewhere like Latvia (a high risk country) so purchasing those numbers was allowed in code but a user manual changed their settings in Console.

I'm not sure what the best approach is but if you were going to start doing a lot of international texting it'd probably makes sense to either have modules/forks or tell clients to split databases. Even just from a regulatory perspective, if a Canadian org is texting they should want CASL compliance enforced in code and something like that is a weird dependency to have loading in a system that's used outside Canada.

[Frydafly]

@ibrand @maxkc do y'all think it's worth adding this process to our documentation?

[ibrand]

@Frydafly not sure what the best distilled piece of advice would be for docs here... maybe that the code can be manually changed to allow for this?

[maxkc]

@Frydafly not sure this is worth "fixing" for default users given how unpredictable and prone to change some of the country based behavior is. If you were to do that at some point I'd definitely recommend putting an off option in the UI and/or shipping support for MaxPrice in that same release.

@ibrand's approach makes sense to me - documenting expected behavior and how to make those small adjustments in code without accidently spending lots of 💸

Where's the best place to put that documentation? If you can point me in that direction I'll can write a first draft this week. Things that come to mind are:

• Couple sentences on why PR numbers are handled differently (countryCode != ISO 3166)
• tl;dr on cost difference
• Suggested change to availablePhoneNumber in code
• Why you should not try to use areaCode or contains for this
• Suggested change to Geo-Permissions in Twilio Console (preventing the Error 21408 issue)
• Note about loading new data re Clarify role of PHONE_NUMBER_CONUNTRY environment variable #1883
• Other trivial expected behavior changes like MMS converting to an embedded URL and increases in Error 21614

Anything I'm missing?

[Frydafly]

Thanks @maxkc for scoping out the components of this documentation!

Looks like you've already found one good place to add a caveat doc/REFERENCE-environment_variables.md

Another spot could be the HOWTO_BUY_NUMBERS_IN_TWILIO.md, which I am trying to change to HOWTO_BUY_NUMBERS_IN_SPOKE.mdwith this PR https://github.com/MoveOnOrg/Spoke/pull/1946
I was trying to update the doc to explain current number buying practices within Spoke, move the current documentation to the bottom (which explains how to purchase numbers in twilio), but perhaps @maxkc could add a section to that document on "how to buy puerto rican numbers/ text PR"? Let me know what you think!

[Frydafly]

given changes to documentation, is this good to close? @maxkc @ibrand