Broken with last CBA update

[jpillora]

The login process is now broken. Hopefully it's just the login to fix, and the rest of the functionality still works!

[jcwillox]

Hi, I'm currently maintaining a Python API for CommBank (not publically released yet). I've just updated it to support the new login process, so hopefully, I can save you guys some time 👍 .

The login process is similar to before except before you are redirected to the homepage (and are logged in) you now need to manually submit a hidden form in the response, then you are taken to the homepage.

CommBank has added an API for accounts now 😀 /retail/netbank/api/home/v1/accounts this returns a JSON object with a list of the accounts, their balances etc. and a link to their transactions page. The parsing for the transactions page has stayed the same, except when navigating to it for the first time you are required to submit a hidden form in the response then you will then be redirected to the transactions page.

The addition of the hidden forms is due to CommBank moving over to OpenID.

[jpillora]

Thanks for the info Josh!

…

On Tue, 17 Dec 2019 at 12:19 am Josh Willox ***@***.***> wrote: Hi, I'm currently maintaining a Python API for CommBank (not publically released yet). I've just updated it to support the new login process, so hopefully, I can save you guys some time 👍 . The login process is similar to before except before you are redirected to the homepage (and are logged in) you now need to manually submit a hidden form in the response, then you are taken to the homepage. CommBank has added an API for accounts now 😀 /retail/netbank/api/home/v1/accounts this returns a JSON object with a list of the accounts, their balances etc. and a link to their transactions page. The parsing for the transactions page has stayed the same, except when navigating to it for the first time you are required to submit a hidden form in the response then you will then be redirected to the transactions page. The addition of the hidden forms is due to CommBank moving over to OpenID. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#17?email_source=notifications&email_token=AAE2X43COM2CY6YUKPZB2PDQY5573A5CNFSM4J2MCG72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG6VTKI#issuecomment-566057385>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE2X4ZJZIB5MYDVQ7XDV33QY5573ANCNFSM4J2MCG7Q> .

[jcwillox]

A little update, CommBank appears to have reverted the login process back to normal, so everything should work as it did before. Along with that, the API endpoint for accounts is no longer available 😢, so guess its back to web scraping 👍.

[jpillora]

Yep, my new code started failing on 2019-12-31 at 4:55 PM

…

On Sat, 4 Jan 2020 at 00:38, Josh Willox ***@***.***> wrote: A little update, CommBank appears to have reverted the login process back to normal, so everything should work as it did before. Along with that, the API endpoint for accounts is no longer available 😢, so guess its back to web scraping 👍. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#17?email_source=notifications&email_token=AAE2X43QU2XWP55WO2X5ZODQ345XFA5CNFSM4J2MCG72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIBEMAI#issuecomment-570574337>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE2X43UZTHV4L5YAF3QWFDQ345XFANCNFSM4J2MCG7Q> .

[jcwillox]

I feel like I should update this thread with the new changes, commbank has re-enabled that API I was talking about, which means you need to handle those hidden forms I mentioned previously.

Accounts API

https://www.commbank.com.au/retail/netbank/api/home/v1/accounts

Sample python code to extract the relevant data.

[
    {
        "name": account["displayName"],
        "bsb": account["number"][:6],  # first 6 digits are the bsb.
        "number": account["number"][6:],  # rest is the account number.
        "balance": account["balance"][0]["amount"],
        "available_balance": account["availableFunds"][0]["amount"],
        "link": account["link"]["url"],
    }
    for account in response.json()["accounts"]
]

I was trying out mitmproxy the other day and was able to glean some of commbank's mobile APIs. Here's what I found:

all requests need the header Authorization: Bearer <token>.
not completely sure where to get this token but I believe you can extract it from the web login process.

all responses from the APIs are JSON formatted.

Get Balances (i.e get accounts)

https://www.commbank.com.au/innovate/SimpleBalance/v1/balance

Get Transactions

Needs the header x-param-map-accountIdentifier: <product_code>+<bsb><account_number>

https://www.my.commbank.com.au/netbank/EnrichedTransactions/v0/transactions

Get Transaction Detail

Each transaction from the ../transactions endpoint has an id field this needs to be used to get the detailed information for a transaction.

https://www.my.commbank.com.au/netbank/EnrichedTransactions/v0/transactions/accountIdentifier-<id>/details

[jpillora]

Awesome, thanks :)

[svict4]

Hey @jcwillox is your python wrapper ready for release by any chance? 🤞

[jcwillox]

@svict4 haha kinda, the code base isn't great as I wrote it a long time ago, but it does work. I spent a bit refactoring just now to make it a little more like jcwillox/up-bank-api. I'll upload the code to GitHub when I get the chance. I don't use commbank that much anymore so it's probably not going to be maintained that well, and currently, it only supports retrieving accounts and the first page of transactions, but maybe someone can improve upon it.

[svict4]

Fair enough @jcwillox Up is the better bank 💪
Was simply looking for some prior art, so thanks for your previous comment on the various endpoints

[balupton]

Maybe https://www.commbank.com.au/Developer/ will be of use, however there is no docs on how to authenticate with the apis

[jcwillox]

It would be great if we could use those but unfortunately, you can't unless you're a business.

To access consumer APIs, you'll need to be accredited by the ACCC and get the customer's consent

For now we are stuck mimicking the NetBank login process and then using the following unofficial API endpoints.

https://www.commbank.com.au/retail/netbank/accounts/api/accounts
https://www.commbank.com.au/retail/netbank/accounts/api/transactions

[jpillora]

I’ve swapped mostly to headless chrome. I use puppeteer to login, then I do a few api calls in the context of the page to get JSON

…

On Thu, 28 Jul 2022 at 7:10 pm Benjamin Lupton ***@***.***> wrote: Maybe https://www.commbank.com.au/Developer/ will be of use, however there is no docs on how to authenticate with the apis — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE2X42TE3IPB3QO2TBZUVTVWJFATANCNFSM4J2MCG7Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[paytah232]

I feel like I should update this thread with the new changes, commbank has re-enabled that API I was talking about, which means you need to handle those hidden forms I mentioned previously.

Accounts API

https://www.commbank.com.au/retail/netbank/api/home/v1/accounts

Sample python code to extract the relevant data.

[
    {
        "name": account["displayName"],
        "bsb": account["number"][:6],  # first 6 digits are the bsb.
        "number": account["number"][6:],  # rest is the account number.
        "balance": account["balance"][0]["amount"],
        "available_balance": account["availableFunds"][0]["amount"],
        "link": account["link"]["url"],
    }
    for account in response.json()["accounts"]
]

I was trying out mitmproxy the other day and was able to glean some of commbank's mobile APIs. Here's what I found:

all requests need the header Authorization: Bearer <token>.
not completely sure where to get this token but I believe you can extract it from the web login process.

all responses from the APIs are JSON formatted.

Get Balances (i.e get accounts)

https://www.commbank.com.au/innovate/SimpleBalance/v1/balance

Get Transactions

Needs the header x-param-map-accountIdentifier: <product_code>+<bsb><account_number>

https://www.my.commbank.com.au/netbank/EnrichedTransactions/v0/transactions

Get Transaction Detail

Each transaction from the ../transactions endpoint has an id field this needs to be used to get the detailed information for a transaction.

https://www.my.commbank.com.au/netbank/EnrichedTransactions/v0/transactions/accountIdentifier-<id>/details

@jcwillox Did you ever work out how to find that token during the login phase? I am scraping in php, and could only see XSRF-TOKEN, but seems not to work?

[jcwillox]

@paytah232 no sorry never looked into those mobile APIs any further. I just pushed the CommBank API wrapper I wrote in Python all those years ago so maybe that will help you, I believe it still works. It sounds like you're having issues with the login so maybe check out this function https://github.com/jcwillox/commbank-api/blob/9ed0f4d1107c28bddb45cc07d6d3be97cbf82178/commbank/client.py#L26-L69.

Here's the repo with the Python API client, it definitely needs some polish, but it might be helpful to people https://github.com/jcwillox/commbank-api.

[paytah232]

@jcwillox Thanks Josh. Seems the hardest thing I'm finding with PHP implementation is scraping the crawler for a form that allows me to post data. From what I can tell, both of these scripts are creating a form and submitting it, but I don't think PHP can do that - at least, I haven't found a way yet, and I haven't found an existing form I can use once I get past login.

https://github.com/jcwillox/commbank-api/blob/0364bfccc03c3c1ff9d70e6529f6908e44bc0f40/commbank/client.py#L51-L56

node-cba-netbank/src/api.js

Lines 164 to 176 in 7feb68b

	const form = Object.assign({}, response.form, {
	// fill the form
	ctl00$ctl00: 'ctl00$BodyPlaceHolder$updatePanelSearch|ctl00$BodyPlaceHolder$lbSearch',
	__EVENTTARGET: 'ctl00$BodyPlaceHolder$lbSearch',
	__EVENTARGUMENT: '',
	ctl00$BodyPlaceHolder$searchTypeField: '1',
	ctl00$BodyPlaceHolder$radioSwitchDateRange$field$: 'ChooseDates',
	ctl00$BodyPlaceHolder$dateRangeField: 'ChooseDates',
	ctl00$BodyPlaceHolder$fromCalTxtBox$field: from,
	ctl00$BodyPlaceHolder$toCalTxtBox$field: to,
	// Add this for partial update
	ctl00$BodyPlaceHolder$radioSwitchSearchType$field$: 'AllTransactions',
	});

I can however get the 40 most recent transactions using the transactions api. I'm wondering if you found a way to pass a query or header to that api that allowed it to handle to/from dates, increase results, or get more results. I wouldn't even know where to start to try that myself.

I have a forked repo where I'm slowly working on this for a PHP implementation here:
https://github.com/paytah232/php-cba-netbank