Adatis BI Blogs

Using R Tools for Visual Studio (RTVS) with Azure Machine Learning

Azure Machine Learning Whilst in R you can implement very complex Machine Learning algorithms, for anyone new to Machine Learning I personally believe Azure Machine Learning is a more suitable tool for being introduced to the concepts. Please refer to this blog where I have described how to create the Azure Machine Learning web service I will be using in the next section of this blog. You can either use your own web service or follow my other blog, which has been especially written to allow you to follow along with this blog. Coming back to RTVS we want to execute the web service we have created. You need to add a settings JSON file. Add an empty JSON file titled settings.json to C:\Users\<your name>\Documents\.azureml. Handy tip: if you ever want to have a dot at the beginning of a folder name you must place a dot at the end of the name too, which will be removed by windows explorer. So for example if you want a folder called .azureml you must name the folder .azureml. in windows explorer. Copy and paste the following code into the empty JSON file, making sure to enter your Workspace ID and Primary Authorization Token. {"workspace":{ "id" : "<your Workspace ID>", "authorization_token" : "<your Primary Authorization Token>", "api_endpoint": "", "management_endpoint": "" }} You can get your Workspace ID by going to Settings > Name. And the Primary Authorization Token by going to Settings > Authorization Tokens. Once you’re happy save and close the JSON file. Head back into RTVS, we’re ready to get started. There are two ways to proceed. Either I will take you line by line what to do or I have provided an R script containing a function, allowing you to take a shortcut. Whichever option you take the result is the same. Running the predictive experiment in R – Line by line With each line copy and paste it into the console. Firstly a bit of setup, presuming you’ve installed the devtools package as described on the github page for the download, load AzureML and connect to the workspace specified in settings.JSON. To do this use the code below: ## Load the AzureML package. library(AzureML) ## Load the workspace settings using the settings.JSON file. workSpace <- workspace() Next we need to set the web service, this can be any web service created in Azure ML, for this blog we will use the web service created in this blog. The code is as follows: ## Set the web service created in Azure ML. automobileService <- services(workSpace, name = "Automobile Price Regression [Predictive Exp.]") Next we need to define the correct endpoint, this can easily be achieved using: ## Set the endpoint from the web service. automobileEndPoint <- endpoints(workSpace, automobileService) Everything is set up and ready to go, except we need to define our test data. The test data must be in the exact same format as the source data of your experiment. So the exact same amount of columns and with the same column names. Even include the column you are predicting, entering just a 0 or leaving it blank. Below is the test data I used: This will need to be loaded into R and then a data frame. To do so use the code below, make sure the path is pointing towards your test data. ## Load and set the testing data frame. automobileTestData <- data.frame(read.csv("E:\\OneDrive\\Data Science\\AutomobilePriceTestData.csv")) Finally we are ready to do the prediction and see the result! The final line of code needed is: ## Send the test data to the web service and output the result. consume(automobileEndPoint, automobileTestData) Running the predictive experiment – Short cut Below is the entire script, paste the entirety of it into top left R script. automobileRegression <- function(webService, testDataLocation) { ## Load the AzureML package. library(AzureML) ## Load the workspace settings using the settings.JSON file. amlWorkspace <- workspace() ## Set the web service created in Azure ML. automobileService <- services(amlWorkspace, name = webService) ## Set the endpoint from the web service. automobileEndPoint <- endpoints(amlWorkspace, automobileService) ## Load and set the testing data frame. automobileTestData <- data.frame(read.csv(testDataLocation)) ## Send the test data to the web service and output the result. consume(automobileEndPoint, automobileTestData) } Run the script by highlighting the whole of the function and typing Ctrl + Enter. Then run the function by typing the below code into the console: automobileRegression("Automobile Price Regression [Predictive Exp.]","E:\\OneDrive\\Data Science\\AutomobilePriceTestData.csv") Where the first parameter is the name of the Azure ML web service and the second is the path of the test data file. The Result Both methods should give you the same result: an output of a data frame displaying the test data with the predicted value: Wahoo! There you have it, a predictive analytic regression Azure Machine Learning experiment running through Visual Studio… the possibilities are endless!

Introduction to R Tools for Visual Studio (RTVS)

Introduction This blog is not looking at one or two exciting technologies, but THREE! Namely Visual Studio, R and Azure Machine Learning. We will be looking at bringing them together in harmony using R Tools for Visual Studio (RTVS). Installation As this blog will be touching on a whole host of technologies, I won’t be going into much detail on how to set each one up. However instead I will provide you with a flurry of links which will provide you with all the information you need. Here comes the flurry…! · Visual Studio 2015 with Update 1 – I hope anyone reading this is familiar with Visual Studio, but to piece all these technologies together version 2015 with Update 1 is required, look no further than here: · R – Not sure exactly what version is needed but just go ahead and get the latest version you can, which can be found here: · Azure Machine Learning – No installation required here, yay! But you will need to set up an account if you have not done so already, this can be done here · R Tools for Visual Studio - More commonly known as RTVS. The name is fairly self-explanatory but it allows you to run R through Visual Studio. If you have used R and Visual Studio separately before it will feel strangely familiar. Everything you need to download, install and set up can be found here: · Devtools Package - The final installation step is a simple one. Installing the correct R packages to allow you to interact with Azure ML. If you’ve used R to interact with Azure ML before you probably have already done this step, but for those who have not, all the information you will need to do so can be found here: Introduction to RTVS Once all the prerequisites have been installed it is time to move onto the fun stuff! Open up Visual Studio 2015 and add an R Project: File > Add > New Project and select R. You will be presented with the screen below, name the project AutomobileRegression and select OK. Microsoft have done a fantastic job realising that the settings and toolbar required in R is very different to those required when using Visual Studio, so they have split them out and made it very easy to switch between the two. To switch to the settings designed for using R go to R Tools > Data Science Settings you’ll be presented with two pop ups select Yes on both to proceed. This will now allow you to use all those nifty shortcuts you have learnt to use in RStudio. Anytime you want to go back to the original settings you can do so by going to Tools > Import/Export Settings. You should be now be looking at a screen similar to the one below: This should look very recognisable to anyone familiar to R:   For those not familiar, the top left window is the R script, this will be where you do your work and what you will run. Bottom left is the console, this allows you to type in commands and see the output, from here you will run your R scripts and test various functions. Top right is your environment, this shows all your current objects and allows you to interact with them. You can also change to History, which displays a history of the commands used so far in the console. Finally the bottom right is where Visual Studio differs from RStudio a bit. The familiar Solution Explorer is visible within Visual Studio and serves its usual function. Visual Studio does contain R Plot and R Help though, which both also feature in RStudio. R Plot will display plots of graphs when appropriate. R Help provides more information on the different functions available within R. Look for my next blog, which will go into more detail on how to use RTVS.

Azure ML Regression Example - Part 3 Deploying the Model

In the final blog of this series we will take the regression model we have created earlier in the series and make it accessible so it can be consumed by other programs. Making the experiment accessible to the outside world The next part of the process is to make the whole experiment accessible to the world outside of Azure ML. To do so you need to create a Web Service. This can be achieved by clicking Set Up Web Service button next to Run and then selecting Predictive Web Service [Recommended]. The experiment will change in front of your eyes and you should be left with a canvas looking similar to the one displayed below. If you would like to get back to your training experiment at any time you can do so by clicking Training experiment in the top right corner. You can then run the predictive web service again to update the predictive experiment. Whilst in the predictive experiment window run the experiment once again and then click Deploy Web Service. Having done this, you should be displayed with the below screen: Select Excel 2013 or later in the same row as REQUEST/RESPONSE. Click the tick to download the Excel document, open it and click Enable Editing, you will see something like the image below. If you are using Excel 2010 feel free to follow the example, it will be fairly similar, but not identical. Click Automobile Price Regression [Predictive Exp.] to begin. Click Use sample data to quickly construct a table with all the appropriate columns and a few examples. Feel free to alter the sample data to your heart’s content. Once you’re happy with your data, highlight it and select it as the Input range selected. Chose an empty cell as the Output. Click Predict. You should see something similar to below: You should now be able to see a replica table with a Scored Labels column displaying the estimated price for each row. Go ahead and rerun the experiment putting in whatever attribute values you desire. This experiment will now always return a Scored Label relating to the price based upon the training model. What next? This has just been a toe dip into the world of Azure ML. For more information on getting started with Azure ML track down a copy of Microsoft Azure Essentials – Azure Machine Learning by Jeff Barnes, this is a great starting point. If you want to know what you can do with Azure ML and how to start using Azure ML within other programs then check out my upcoming blog which will show you how to integrate Azure ML straight into Visual Studio.

SQL PASS Summit - Day 3

Today was the final day of the pass summit. The schedule was a little different in the morning with no keynote speech. Instead I opted to see Jerimiah Peschka’s session ‘Dynamic SQL: Build Fast, Flexible Queries’. This was an interesting talk, which covered the security aspects of dynamic SQL and how to avoid things such as SQL injection and also how to ensure that queries generated from dynamic SQL ran quickly and avoided issues such as ensuring plan reuse on each execution. Whilst I very rarely find a myself situation in which Dynamic SQL is essential I found this an interesting talk and I am sure that when the need does arise it will have been very handy. Following this I attended a Microsoft lab on Azure Machine Learning (AML) ‘Authoring a hand-written digit recognizer in Microsoft Azure Machine Learning Studio’. This was a hands on session where computers and a set of exercises and sample data are supplied. Essentially the lesson consisted of providing a model with vector images of numbers, training the model to recognise the numbers and then passing a new set of number vector images to the model to see it identify them (> 85%) correctly. The session was interesting and gave me an overview of using AML Studio which was great. My main issue was that the data was already provided in vector format (csv files which were imported), whilst I can see time constraints mean that the conversion from image to vector during the class would have been difficult it would have been very interesting to have seen a few of the examples of the images so that we could see exactly what it was that was being identified and how different the numbers were as this would give an illustration of how clever the algorithms are. I finished the conference with Brain Larson’s talk entitled ‘Effective Reporting through SSRS Advanced Authoring Features’. As you may know SSRS has not been updated for some time now and with the current focus on Power View I wasn’t sure how much I would benefit from this and what the plans are for this tool in the future. I was quite surprised therefore when the room filled up and overflowed so that there were quite a crowd of people in the room who had to stand in order to watch it. The talk showed some interesting uses of SSRS - images embedded in the background of reports, adding strip lines and markers and an example of creating an interactive report which allowed users to set their preferences for running other reports. The examples given were very good as was the delivery, my only issue here is that without new features added to SSRS (for example ability to do an asynchronous refresh of some report elements) I am not sure what the uptake of SSRS is going to be. All in all I have had a fantastic and informative time here. I leave you with some pictures, taken earlier in the week :)