iOS7 Support and Discussion

[plamoni]

UPDATE:

Thanks for everyone's continued interest in SiriProxy. I know lots of people are really excited to get it "up and running" with iOS7. Now that the software is public, you're welcome to discuss bugs/issues on on this forum. However, keep two things in mind:

1: Only public releases of iOS are supported by SiriProxy
I don't have a developer account with Apple right now, so I'm not under NDA (I never had access to pre-release iOS7), but my understanding is that the pre-release versions of the software point to a different set of servers from the production versions. Therefore, you'll have a number of issues as guzzoni.apple.com is hard-coded in in several places (and is an integral part of the SSL MITM cert generation process). This being the case, I highly recommend upgrading to the public release (NOT the GM release from earlier this week) before you file any bugs with SiriProxy.

2: SiriProxy has no full time development staff
At this point, I'm the sole developer on SiriProxy. ElvisImpersonator does an AMAZING job at supporting the SiriProxy developer community, but doesn't work on the core development. Both of us have full time jobs that keep us extremely busy and neither of us get paid to work on SiriProxy. It's a labor of love (one that you're free to participate in through pull requests!).

That being said, I am on call this week and Elvis has, um, left the building (he's traveling for work), so it might be a week or more before I even get a chance to do testing on iOS7. So, as with before: UPDATE TO iOS7 AT YOUR OWN RISK. There's no official support for iOS7 in SiriProxy at this time and I can't make any guarantees it will ever work on iOS7. As with any new release of iOS, Apple could (any may already have) put in measures to block SiriProxy from working.

Again, thank you for your continued interest in my project. SiriProxy started out as a fun short-term project that kept me busy while I was between jobs and has turned into much more than I ever would have expected. I will continue to try and keep things working as long as I can and I'm always every excited to see the awesome projects that you, the SiriProxy development community, create! Keep up the great work and thanks again!!!

-Pete

---

Original:

Okay, so I'm watching the WWDC right now and there's lots of change to Siri coming. This means two things:

1: NEW BETA -- NO DISCUSSION

There's a beta out today for developers. Please remember that as an Apple Developer, you must agree to an NDA regarding unreleased applications (e.g. iOS7). This means that you should refrain from discussion of changes to the Siri protocol or functionality until iOS7 is publicly released. Once it's out for the public we can start working on fixing whatever it might break. Please do not open issues regarding problems when using iOS7 until it is publicly released. Elvis and I will close these tickets.

2: SiriProxy might break with beta software

As mentioned, a new iOS might mean changes to the Siri protocol. The iOS6 changes were minimal and I was able to fix them pretty quickly. No promises for iOS7 or future software. Hopefully it's something I'll be able to work with, but don't go upgrading devices if SiriProxy is important to you.

Thanks everyone for your interest!

-Pete

[plamoni]

I really did not want this to turn into a focal point for violations of the developer NDA. Please do not discuss unreleased Apple software on this forum. For now on I'm just going to delete any such posts without warning or modification.

[elvisimprsntr]

Do not upgrade your devices to iOS 7 GM Seed if SiriProxy is important to you. Please keep in mind that iOS 7 is not yet a public release and we are still bound by an iOS developer NDA. We are investigating any issues for possible solutions. We will provide updates when we have relevant information we can disclose.

[alexrmc92]

Now that iOS 7 has officially been released lets get this working. It's been a few months since i've looked at the source. I don't think iOS7 resolve to guzzoni.apple.com anymore. With my local DNS server handling that domain, siri will still function on iOS7. Perhaps they changed the servers DNS name, or possibly they internalized the ip address for it. They could have also set iOS7 to use a specific name server for Siri.

I'll do some packet tracing at work today and see what i can find out.

[elvisimprsntr]

I know you are trying to be helpful, but please refrain from posting conjecture or speculation. It does not serve anyone any good and dilutes resources from real solutions. Pete is already looking into it.

P.S. I'll be sure to let your boss know you are using company resources and time to diagnose the problem. ;)

[Sikozu]

Guys I came across this while looking for a solution to get siri-proxy working whilst I was in the beta;

zhangyuanwei/node-siri#13

The server is now daryl.apple.com, which I got also from my own tests.

[elvisimprsntr]

I know everyone is anxious and trying to be helpful, but once again please refrain from conjecture and speculation. What someone may have discovered while iOS was in beta is not likely to apply to the public release. From now on any information or posts that are known to be inaccurate will automatically be deleted.

[fablanglet]

The struture of the siri's object have been changed. "Properties" doesn't exist now.
The property "class" become "$class", for "group" and "v" it's the same case. See the example below.
Moreover, the domain daryl.apple.com has been used for the beta version, now it's ever guzzoni.apple.com but the version change, and the user-agent change
IOS 6.1: User-Agent: Assistant(iPhone/iPhone4,1; iPhone OS/6.1.3/10B329) Ace/2.1
IOS 7: User-Agent: Assistant(iPhone/iPhone4,1; iPhone OS/7.0/11A465) Ace/3.0

It's my observation, i modified the siriproxy code, but the protocol have more changes, because even i modified the structure of the object, siriproxy doesn't work. If someone has more information, let me know and we can work together (@Fab_Lan or html5webapp.blogspot.com).

Example for SpeechRecognized (IOS7):
{"$group"=>"com.apple.ace.speech",
"$class"=>"SpeechRecognized",
"$v"=>"3.0",
"aceId"=>"GUID",
"refId"=>"GUID",
"recognition"=>
{"$group"=>"com.apple.ace.speech",
"$class"=>"Recognition",
"$v"=>"3.0",
"phrases"=>
[{"$group"=>"com.apple.ace.speech",
"$class"=>"Phrase",
"$v"=>"3.0",
"interpretations"=>
[{"$group"=>"com.apple.ace.speech",
"$class"=>"Interpretation",
"$v"=>"3.0",
"tokens"=>
[{"$group"=>"com.apple.ace.speech",
"$class"=>"Token",
"$v"=>"3.0",
"confidenceScore"=>804.0,
"endTime"=>780,
"originalText"=>"Teste",
"removeSpaceAfter"=>false,
"removeSpaceBefore"=>true,
"startTime"=>0,
"text"=>"Teste"}]}],
"lowConfidence"=>false,
"speechRepairApplied"=>false},
{"$group"=>"com.apple.ace.speech",
"$class"=>"Phrase",
"$v"=>"3.0",
"interpretations"=>
[{"$group"=>"com.apple.ace.speech",
"$class"=>"Interpretation",
"$v"=>"3.0",
"tokens"=>
[{"$group"=>"com.apple.ace.speech",
"$class"=>"Token",
"$v"=>"3.0",
"confidenceScore"=>808.0,
"endTime"=>1080,
"originalText"=>"Siri",
"removeSpaceAfter"=>false,
"removeSpaceBefore"=>false,
"startTime"=>780,
"text"=>"Siri"}]}],
"lowConfidence"=>false,
"speechRepairApplied"=>false},
{"$group"=>"com.apple.ace.speech",
"$class"=>"Phrase",
"$v"=>"3.0",
"interpretations"=>
[{"$group"=>"com.apple.ace.speech",
"$class"=>"Interpretation",
"$v"=>"3.0",
"tokens"=>
[{"$group"=>"com.apple.ace.speech",
"$class"=>"Token",
"$v"=>"3.0",
"confidenceScore"=>883.0,
"endTime"=>2780,
"originalText"=>"proxy",
"removeSpaceAfter"=>false,
"removeSpaceBefore"=>false,
"startTime"=>1080,
"text"=>"proxy"}]}],
"lowConfidence"=>false,
"speechRepairApplied"=>false}],
"sentenceConfidence"=>514},
"sessionId"=>"GUID"}

[bgottsch]

I've noticed 2 different headers with the new iOS 7 (I run the non-beta version). Maybe this can help someone. Also, in another test, I changed some of the SiriProxy code and had what I believe were problems with the SSL connection with the iDevice.

(the log below is from the non-modified SiriProxy server and in this test I was able to get a response from Siri)

[Info - Server] DNS Server started, tainting 'guzzoni.apple.com' with 10.0.1.25
[Info - Server] Starting SiriProxy on 0.0.0.0:443...
[Info - Server] SiriProxy up and running.
[Info - Plugin Manager] Plugins loaded: Example, Computer, Personal
[Info - iPhone] SSL completed for iPhone
[Header - iPhone] ACE /ace HTTP/1.0
[Header - iPhone] Host: guzzoni.apple.com
[Header - iPhone] Content-Length: 2000000000
[Header - iPhone] User-Agent: Assistant(iPhone/iPhone5,1; iPhone OS/7.0/11A465) Ace/3.0
[Header - iPhone] Accept-Language: en-US
[Header - iPhone] X-Ace-Host: st14p02sa:405549ce-fcc8-4779-b412-1ddf9eaacfed
[Header - iPhone]
[Debug - iPhone] Found end of headers
[Info - Plugin Manager] Plugins loaded: Example, Computer, Personal
[Info - iPhone] SSL completed for iPhone
[Header - iPhone] ACE /ace HTTP/1.0
[Header - iPhone] Host: guzzoni.apple.com
[Header - iPhone] Content-Length: 2000000000
[Header - iPhone] User-Agent: Assistant(iPhone/iPhone5,1; iPhone OS/7.0/11A465) Ace/3.0
[Header - iPhone] Accept-Language: en-US
[Header - iPhone] X-Ace-Host: st14p02sa:405549ce-fcc8-4779-b412-1ddf9eaacfed
[Header - iPhone]
[Debug - iPhone] Found end of headers
[Info - iPhone] Received Object: (group: , ref_id: , ace_id: F24C73B3-0681-476B-AAC5-1126D671BE26)
{"$class"=>"LoadAssistant",
"connectionType"=>"Wifi",
"speechId"=>"539335ad-021b-40a4-9496-b74e8c1806a5",
"assistantId"=>"a1746c70-7c6e-4dea-8401-e43f4c8eaa9f",
"activationToken"=>
"\xB1\xA2\xF6\xE6\xE2\x10\xC5 >j\xBD\x126\xA1\x97 \xB5j$",
"aceId"=>"F24C73B3-0681-476B-AAC5-1126D671BE26",
"$group"=>"com.apple.ace.system",
"sessionValidationData"=>
"\x02\xB5\xEB\xDB\x7F\x84\xA1{\xAE\xD3\x02s\x8B5\xC1\x05\xA4\xA2iT\xAF\x9D\x83V\xA0f\xB8L\x83c$\xF5h\x00\x00\x02\x00\x06\x00\x00\x00O\x00\x00\x00\x80\xEF\xAEP\xB1\xF06\xD1\xDD\xF0\x9D\x8E\xD3\xDA\xDB\x86\x05T\xEB\xFA\x1D\x8B\xA6\xB9z\b\xEB\x10.\x04'F\x11\xDDZ\x02\t\x96j\xEDJ\x90\xA2R\xDC\xB7\x03\x1ADo\xC5\xD2\xCD\x1EK\xD6\xE2&\xC6\xAB\xCE\xBE\x00]-\xA71=Bb\xA5\xE9\x98\e\xE7\xD2s!\xD0\xE7\xD2\xD6Q\xA2\x92Yw\xDF\x13W1\x9EH{\xD7P\xA6\x00\xFEM\xC3dG\r\x14\xEC1\x9D\xE3I\xB8\xEE/\x11[\xFB\f\f @m\xC4]\xCC'\x0F\x95\xA6\x00\x00\x01\x14\n\n<plist version="1.0">\n\n\tAssistant Identifier\n\ta1746c70-7c6e-4dea-8401-e43f4c8eaa9f\n\n\n\x00\x00\x00O\x01Lg\xA96\x93CA\xC8\a7%L\x90\xA2\xCE\xA5Z\x90\x02\x0F\x00\x00\x006\x02\x02H\xC5J\x95JR\xF6\xD2+x\n\xC9J\x19\xD0kR;\xB7#\x9D\xD1\xF9\xA4<\x8A\xD68(\xEA\xD4\x92Y\xD4YXq9\x1A\xDC\xFF{a\x15\x1D\x95\xE3\xA2\xE3c\xE57\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00",
"language"=>"en-US"}
[Info - Forwarding object to Guzzoni]
[Info - iPhone] Received Object: (group: , ref_id: , ace_id: 4FCE4B6C-B901-4EB0-83D1-DF2FDAB5DF7B)
{"$class"=>"SetRestrictions",
"$group"=>"com.apple.ace.system",
"aceId"=>"4FCE4B6C-B901-4EB0-83D1-DF2FDAB5DF7B"}
[Info - Forwarding object to Guzzoni]
[Info - iPhone] Received Object: (group: , ref_id: , ace_id: C91C405F-4D4A-4B66-9455-956907C5D702)
{"$class"=>"ClearContext",
"$group"=>"com.apple.ace.system",
"aceId"=>"C91C405F-4D4A-4B66-9455-956907C5D702"}
[Info - Forwarding object to Guzzoni]
[Info - iPhone] Received Object: (group: , ref_id: , ace_id: BB62AF68-CEAF-4FE2-9462-D2AEFD9EA5B2)
{"aceId"=>"BB62AF68-CEAF-4FE2-9462-D2AEFD9EA5B2",
"$class"=>"UpdateRestrictions",
"$group"=>"com.apple.ace.system",
"restrictionsToRemove"=>["STARK"]}
[Info - Forwarding object to Guzzoni]
[Info - Guzzoni] SSL completed for Guzzoni
[Info - Guzzoni] SSL completed for Guzzoni
[Info - Plugin Manager] Plugins loaded: Example, Computer, Personal
[Info - Plugin Manager] Plugins loaded: Example, Computer, Personal
[Info - iPhone] SSL completed for iPhone
[Info - iPhone] SSL completed for iPhone
[Header - iPhone] HEAD /salt HTTP/1.1
[Header - iPhone] Host: guzzoni.apple.com
[Header - iPhone] Accept-Encoding: gzip, deflate
[Header - iPhone] Accept: /
[Header - iPhone] Content-Length: 0
[Header - iPhone] Connection: keep-alive
[Header - iPhone] Accept-Language: en-us
[Header - iPhone] User-Agent: AssistantServices/1 CFNetwork/672.0.2 Darwin/14.0.0
[Header - iPhone]
[Debug - iPhone] Found end of headers
[Header - iPhone] HEAD /ace HTTP/1.1
[Header - iPhone] Host: guzzoni.apple.com
[Header - iPhone] Accept-Encoding: gzip, deflate
[Header - iPhone] Accept: /
[Header - iPhone] Content-Length: 0
[Header - iPhone] Connection: keep-alive
[Header - iPhone] Accept-Language: en-us
[Header - iPhone] User-Agent: AssistantServices/1 CFNetwork/672.0.2 Darwin/14.0.0
[Header - iPhone]
[Debug - iPhone] Found end of headers
[Info - Guzzoni] SSL completed for Guzzoni
[Info - Guzzoni] SSL completed for Guzzoni

[bgottsch]

Also another issue I discovered was that Siri on my iPhone, when connect to the SiriProxy over wifi, uses cellular data to get the requests. When I have cellular + wifi enabled Siri works and presents 2 different headers. When I disable it (wifi only), it shows only one header and does not work. I tested it on my wifi iPad and I get the same result as the iPhone with cellular off, and in this log it appears to be doing circles, differently from the one I posted above. (the above one was using wifi + 3g)

[elvisimprsntr]

Siri was known to "leak" over the cellular connection in iOS 6 when the cellular connection was determined to be better, which was often the case for me when connected to LTE. Thus one had to disable the cellular data connection.

I came across this quote by Olivier Bonaventure, one of, if not the leading researcher in the field of Multi Path TCP.
Olivier Bonaventure writes:
"Besides changes in UI, multitasking and other features that the press discusses, iOS7 also includes support for Multipath TCP. Multipath TCP is a major extension to TCP that is able to use different interfaces for the same connection. Until now, Multipath TCP has been mainly used by researchers with a modified Linux kernel. iOS7 changes that, with millions of Multipath-TCP enabled devices that can switch from 3G to WiFi without losing existing TCP connections. This is not yet the case on iOS7, which currently seems to only enable it for SIRI, but other use cases will likely appear in the future."
Source: http://m.slashdot.org/story/191809

I am not an expert, but MPTCP seems to offer multiple connections over multiple IP addresses (e.g. Cellular, Wifi) to provide more robust services when one connection quality is poor or is dropped. While these reports suggest Apple is limiting MPTCP between Wifi and cellular and only for Siri, I can see the benefits of providing more robust connection to services when switching cell towers, when for a brief moment your phone has multiple cellular data connections. Also could be a used for persistent MPTCP LTE connections to improve throughput and robustness.

Regardless, from what little I have read on MPTCP, it seems turing off the cellular connection to limit Siri to using Wifi only would thwart Siri trying to use both.

[bgottsch]

@elvisimprsntr Maybe this could represent a problem. I'm looking into MPTCP. Therefore, it is possible that the iPhone prefers wifi over cellular, so if the SiriProxy connection is working it won't be a problem. Another way I though to fix this is to use VPN. While I using iOS 6, I used VPN over cellular and it worked perfectly with SiriProxy. Although, with or without the MPTCP, SiriProxy is yet not able to work with iOS 7. I think the MPTCP problems will be better figured out when it is working. For now we will have to disable cellular while we try to find a fix to SiriProxy. When I have more info of how Siri on iOS 7 works I'll let you know. In the mean time, I'll have a look into MPTCP.

[elvisimprsntr]

@bgottsch I used a VPN to invoke SiriProxy in iOS 6 with success as well. At the moment I am having trouble getting Siri to route thru the VPN connection. It seems Siri ignores the "route all traffic thru VPN" setting, or at least the symptoms appear that way.

[bgottsch]

@elvisimprsntr I just tested it using VPN (I run iVPN) over 3G and the SiriProxy identifies a connection from my iPhone. I found 2 headers in the log:

[Header - iPhone] ACE /ace HTTP/1.0
[Header - iPhone] Host: guzzoni.apple.com
[Header - iPhone] Content-Length: 2000000000
[Header - iPhone] User-Agent: Assistant(iPhone/iPhone5,1; iPhone OS/7.0/11A465) Ace/3.0
[Header - iPhone] Accept-Language: en-US

and

[Header - iPhone] HEAD /ace HTTP/1.1
[Header - iPhone] Host: guzzoni.apple.com
[Header - iPhone] Accept-Encoding: gzip, deflate
[Header - iPhone] Accept: /
[Header - iPhone] Content-Length: 0
[Header - iPhone] Connection: keep-alive
[Header - iPhone] Accept-Language: en-us
[Header - iPhone] User-Agent: AssistantServices/1 CFNetwork/672.0.2 Darwin/14.0.0

When I ask Siri something over this VPN connection it does not respond, so I think it is working. On my VPN setup, I point my computer running SiriProxy as the DNS server and run RubyDNS from the SiriProxy on the same address. Maybe this is preventing your iPhone to connect to the server.

Also I have found this link http://www.networkworld.com/news/2013/091913-ios7-multipath-273995.html which says "MPTCP is spreading the connection’s data across several subflows, sending it over the least congested paths."

[elvisimprsntr]

@bgottsch I use DD-WRT for both my VPN server and to perform a transparent guzzoni.apple.com redirect, which worked well under iOS 6.

The source of the information all points to Professor Olivier Bonaventure, http://perso.uclouvain.be/olivier.bonaventure/blog/html/2013/09/18/mptcp.html
I wonder if he actually consulted Apple and once the "Cone of Silence" was lifted when iOS 7 became public, allowed him to publish Apple's use of MPTCP. If Apple is using his implementation of MPTCP, it would certainly help his case over the competing implementations.

Regardless, it seems like getting SiriProxy working with iOS 7 will be more work than anticipated. I see it as a challenge and opportunity to learn.

[bgottsch]

@elvisimprsntr It seems that iOS 7 avoids using VPN to connect Siri. I am having problems with the VPN connection and Siri. It appears to be avoiding, as you said before. The DNS + wifi-only solution is working. For now I'll use that to see whats happening to SiriProxy. I agree, it will be a challenge.