Prosper recently re-released their API, completely rewritten from the SOAP/WSDL service to a REST compliant service. This is an exciting change for three reasons:
1.) More open. The move to REST will allow greater access to Prosper’s API. Not only can you access data directly from your web browser but I believe developers will find the API much easier to work with too. When you make a request to their API can you specify your content type as XML (default) or JSON which gives developers greater flexibility to consume the data in a format of their choice. There is a lot of freedom in terms of how you process the service results.
2.) Richness in data. Prosper has included endpoint access to your account balances and notes. These are wonderful additions, but it doesn’t stop there. They have a new endpoint called DataDictionary which states every data element, the description, and SQL type for the specified endpoint. One of the biggest pains when handling a large amounts of data with no definition of the types is guessing what is most reasonable to use for your SQL schema. Not only is this solved, but with the SQL types returned by the service. You can create scripts to automatically generate and maintain your schema for Prosper data. This will translate to saved time and ease when maintaining your local copy of Prosper data.
3.) Security. Prosper requires a separate set of credentials to access the API. This means applications will use a set of credentials that can only access API functions, and not your main account which you log into from Prosper.
Below is a brief descriptions of the new Endpoints:
Listings: GET https://api.prosper.com/api/Listings/
Listings contain 541 data elements per loan as of writing this. A bulk of these are part of the credit pull they receive from their expanded underwriting process.
ListingsHistorical: GET https://api.prosper.com/api/ListingsHistorical/
This is the same data contained in Listings, but for all listings that have ever existed. It’s a very large data set, and we’ll discus later how to “chunk” the data into smaller downloads. But the best way to get this data will likely still be direct from Prosper’s data download page.
Invest: POST https://api.prosper.com/Invest
The Invest endpoint will allow you to place an order for a note. Typically POSTs aren’t something you can do in a web browser manually but plugins like REST Console (covered later) would allow you to do so. For developers its important to set your Content-Type to application/x-www-form-urlencoded for the investment to work correctly.
To do the invest all you need to do is POST two parameters:
- listingId: the listing number of note you wish to buy
- amount: The amount you wish to buy
If you are trying to buy $100 of a loan that only has $50 left, the service will automatically round down and tell you in the response how much of the investment amount was actually invested.
Investments: GET https://api.prosper.com/api/Investments/
Investments is an interesting endpoint because is allows you to see a detailed audit trail of all investment history for your account, not just notes that have been issued. It would be very easy to reconstruct your “issue rate” to see how many of your loans are actually getting issued. This is a great metric, and one I personally monitor for my investments.
Notes: GET https://api.prosper.com/api/Notes/
Notes will be critical for those who in the past relied on LendStats or ProsperStats to gain insight into their portfolio. This is essentially the Lending Club equivalent to notes.csv. In addition to showing you all the notes you own, Prosper gives you some nice additional data such as:
- Has the note been sold on FolioFN
- A complete breakdown of fees
- Recovery amounts
- Days past due
Later in the blog post I will show you how to get your Notes from your web browser in CSV format.
Loans: GET https://api.prosper.com/api/Loans/
The Loans endpoint provides a high level overview of loan performance. It won’t contain expanded underwriting, but will contain vital information related to balances. In many ways it shares a lot of same data elements as the Notes endpoint, but include all loan, not just ones you’ve invested in.
MarketPlace: GET https://api.prosper.com/api/MarketPlace/
MarketPlace is not yet implemented yet, but will contain data on the overall platform performance. Such as total issued, interest rates, and other aggregate data points.
Account: GET https://api.prosper.com/api/Account/
Account is not yet implemented. There is no public documentation on this endpoint, but from what I have been told it will allow you to access information such as idle cash, and total investment in terms of outstanding principle and pending notes.
oData is not an Endpoint, but is method for filtering data. This allows you to specify subsets of data from the API. For instance you may want to look at only A grade loans for Listings, or find loans in a specific interest rate range. Theoretically, developers could rewrite their filters using oData, make an API call to Listings and simply buy the loans returned. No need for client side filtering or even storing data!
A good example would be using oData to filter ListingsHistory for all issued loans. Since ListingsHistory also contains loans that were not issued, we will have to filter these out.
GET https://api.prosper.com/api/ListingsHistorical/?$filter=LoanOriginationDate ne null
We are searching for all loans where the origination date is not NULL – NULL indicates the loan was not issued.
Keep in mind, this is a huge set of data. I would recommend using oData to break this request into smaller ones using $skip and $top to implement paging.
- Request 1: https://api.prosper.com/api/ListingsHistorical/?$filter=LoanOriginationDate ne null&$top=50&$skip=0
- Request 2: https://api.prosper.com/api/ListingsHistorical/?$filter=LoanOriginationDate ne null&$top=50&$skip=50
- Request n: 50(n-1): https://api.prosper.com/api/ListingsHistorical/?$filter=LoanOriginationDate ne null&$top=50&$skip=50(n-1)
You would do requests until you result set is a size of 0. At that point you would have a complete data set for all issued loans.
How to Access the API (Even if you’ve never written a single line of code in your life!)
One of the advantages of a REST based service is the simplicity. REST uses the same protocol the world wide web uses, HTTP, which means your browser can act as a client for the API. You can you can start accessing data with ease and speed. For all of the service endpoints that support GET, you can go directly to the endpoint in your browser and retrieve data in XML format. However, the API supports CSV and JSON if you specify this in your Content-Type. A quick summary of the support contents type are:
Please remember, in order to access the API you must contact Prosper’s Investment Services.
Using Chrome’s Rest Console to Access Your Notes
Chrome has a plugin called REST Console which allows you to interact with REST services using a graphical interface. By setting a few parameters you can start accessing the data without having to write a single line of code.
1. Install & start REST Console from inside Chrome
2. Configure the Target
3. Set your credentials for the API
4. Click Send
You will see your currently owned notes returned in the console output. You can get the exact same data without using REST Console but it will be in XML format.
The new API offering for Prosper is a game changer. The DataDictionary endpoint is probably the least interesting in terms functionality but will provide developers with the ability to create applications in less time and with more robustness. The DataDictionary also contains the description for all of the expanded underwriting elements so you will know what the data means and how it may impact your investing. As more developers decide to automate their own investing, it will prove and invaluable resource.
As for the other Endpoints, Prosper has literally exposed virtually all data they have for their loans through their new API. There is no doubt this will be impactful on investors and open all sorts of new discussion on how we use data to invest.