From 8ff340c1105ad9a4ec589d026f540a5a9e8539b1 Mon Sep 17 00:00:00 2001
From: solmazeradat-aridhia
 <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Thu, 4 Nov 2021 15:59:41 +0000
Subject: [PATCH 01/22] tested step1 and 2 and updated readme

---
 src/data-profiler/README.md | 29 +++++++++++++++++++++--------
 1 file changed, 21 insertions(+), 8 deletions(-)

diff --git a/src/data-profiler/README.md b/src/data-profiler/README.md
index 3b22cf7..ec50d2c 100644
--- a/src/data-profiler/README.md
+++ b/src/data-profiler/README.md
@@ -2,15 +2,23 @@
 
 ## Overview
 
-This is a simple worked example of a summarisation task that can be wrapped up as a docker container. It is intended to work on any tabular data it finds in the input folder specified. This means it can be used for data explorations: it makes few assumptions on the source data other than it being valid CSV.
+This is a simple worked example of a summarisation task that can be wrapped up as a docker container. To see a workflow diagram of the steps taken below go to [Containerising a script as a federated compute task](https://github.com/federated-data-sharing/common-api/blob/master/doc/User_Guide_Containerising_Tasks.md#containerising-a-script-as-a-federated-compute-task).
 
-The script itself [data-profiler.py](./data-profiler.py) used the [pandas](https://pandas.pydata.org/) library to create a brief statistical profile of each field in the source data and write these to an output file.
+This example is intended to work on any tabular data it finds in the input folder specified. This means it can be used for data explorations: it makes few assumptions on the source data other than it being valid CSV. 
 
-## Step-by-step
+The script itself [data-profiler.py](./data-profiler.py) uses the [pandas](https://pandas.pydata.org/) library to create a brief statistical profile of each field in the source data and write these to an output file.
 
-Create an `input` and an `output` folder here. Put one or more CSV files in the `input` folder.
+## Step 1: Run script locally on command line
 
-Run the script directly, on the command line:
+- Copy the repository on to your local machine. 
+
+- Create an `input` and an `output` folder under the directory ```.../src/data-profiler```. 
+
+![image](https://user-images.githubusercontent.com/91956839/140361598-e4eb71b2-f058-457c-9066-93022acb5e48.png)
+
+- Put one or more CSV files in the `input` folder.
+
+- Run the script directly, on the command line:
 ```sh
 rm output/*
 
@@ -21,14 +29,17 @@ python data-profiler.py
 ```
 Look at the output files to see the statistical summaries.
 
-Then, build the docker image
+## Step 2: Run containeised script via docker commandline 
+
+- Build the docker image
+
 ```sh
 docker build . -t data-profiler
 ```
 
 > Depending on your docker set up you may need to run this command prefixed by `sudo`
 
-Then run the container on the same local file:
+- Then run the container on the same local file:
 ```sh
 rm output/*
 
@@ -37,11 +48,13 @@ docker run -it\
      --mount type=bind,source="`realpath $(pwd)/output`",target=/mnt/output\
      data-profiler:latest
 ```
+Look at the output files to see the statistical summaries.
 
 > These commands are also provided as shell scripts
 
-Running this as as a federated data sharing task
+## Step 3: Run containerised script via federated data sharing task
 
 > TODO
 
+https://github.com/federated-data-sharing/common-api/blob/master/doc/User_Guide_Containerising_Tasks.md#containerising-a-script-as-a-federated-compute-task
 

From ff1ef5fa244f8c9e0c6582dd69740b9c3da70e1c Mon Sep 17 00:00:00 2001
From: solmazeradat-aridhia
 <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Thu, 4 Nov 2021 16:07:26 +0000
Subject: [PATCH 02/22] fixed typo

---
 src/data-profiler/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/data-profiler/README.md b/src/data-profiler/README.md
index ec50d2c..2cff873 100644
--- a/src/data-profiler/README.md
+++ b/src/data-profiler/README.md
@@ -29,7 +29,7 @@ python data-profiler.py
 ```
 Look at the output files to see the statistical summaries.
 
-## Step 2: Run containeised script via docker commandline 
+## Step 2: Run containerised script via docker commandline 
 
 - Build the docker image
 

From 364d6f9082b9a3539e51f850418ab65b077f30db Mon Sep 17 00:00:00 2001
From: solmazeradat-aridhia
 <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Thu, 4 Nov 2021 17:00:44 +0000
Subject: [PATCH 03/22] removed link at the bottom of page.

---
 src/data-profiler/README.md | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/src/data-profiler/README.md b/src/data-profiler/README.md
index 2cff873..90f08e8 100644
--- a/src/data-profiler/README.md
+++ b/src/data-profiler/README.md
@@ -55,6 +55,3 @@ Look at the output files to see the statistical summaries.
 ## Step 3: Run containerised script via federated data sharing task
 
 > TODO
-
-https://github.com/federated-data-sharing/common-api/blob/master/doc/User_Guide_Containerising_Tasks.md#containerising-a-script-as-a-federated-compute-task
-

From e61c5901ac223836f1cf7ddf9882a261ac7a760e Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <solmaz.eradat@aridhia.com>
Date: Mon, 6 Dec 2021 13:58:41 +0000
Subject: [PATCH 04/22] create count_null to show number of NaN in each column

---
 src/data-profiler/data-profiler.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/src/data-profiler/data-profiler.py b/src/data-profiler/data-profiler.py
index 87ab949..79c0bfc 100644
--- a/src/data-profiler/data-profiler.py
+++ b/src/data-profiler/data-profiler.py
@@ -51,6 +51,9 @@
             if 'top' in all_summary.columns:
                 all_summary = all_summary.drop(columns=['top']) 
 
+            # count number of missing/NaN values in each column and add to all_summary
+            all_summary['null_count'] = df.isnull().sum(axis = 0)
+
             # write out to output_folder
             all_summary_path = f'{output_folder}/{basename}_summary.csv'
 

From 6240d648579a7ffb7ef768e1b8fc0e0b1070383c Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Mon, 6 Dec 2021 14:30:46 +0000
Subject: [PATCH 05/22] added diagram of summary output

---
 src/data-profiler/README.md | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/src/data-profiler/README.md b/src/data-profiler/README.md
index 90f08e8..ad14268 100644
--- a/src/data-profiler/README.md
+++ b/src/data-profiler/README.md
@@ -8,6 +8,11 @@ This example is intended to work on any tabular data it finds in the input folde
 
 The script itself [data-profiler.py](./data-profiler.py) uses the [pandas](https://pandas.pydata.org/) library to create a brief statistical profile of each field in the source data and write these to an output file.
 
+The summary result of from each cvs file should have the following structure: 
+![image](https://user-images.githubusercontent.com/91956839/144863924-a45cf273-6b82-49f6-bde9-9d1068d11d46.png)
+
+
+
 ## Step 1: Run script locally on command line
 
 - Copy the repository on to your local machine. 

From 276812258ae0ee3aaa051797e5c82bb4574417f6 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Mon, 6 Dec 2021 14:34:08 +0000
Subject: [PATCH 06/22] fixed typos

---
 src/data-profiler/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/data-profiler/README.md b/src/data-profiler/README.md
index ad14268..392c420 100644
--- a/src/data-profiler/README.md
+++ b/src/data-profiler/README.md
@@ -8,7 +8,7 @@ This example is intended to work on any tabular data it finds in the input folde
 
 The script itself [data-profiler.py](./data-profiler.py) uses the [pandas](https://pandas.pydata.org/) library to create a brief statistical profile of each field in the source data and write these to an output file.
 
-The summary result of from each cvs file should have the following structure: 
+The summary result from each csv file should have the following structure: 
 ![image](https://user-images.githubusercontent.com/91956839/144863924-a45cf273-6b82-49f6-bde9-9d1068d11d46.png)
 
 

From 73608e31d96c7b864880c21b3e5e595127422442 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Mon, 6 Dec 2021 14:37:47 +0000
Subject: [PATCH 07/22] Changed wording for consistency with data-profiler

---
 src/data-charts/README.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/data-charts/README.md b/src/data-charts/README.md
index ccf4e18..20dbc40 100644
--- a/src/data-charts/README.md
+++ b/src/data-charts/README.md
@@ -1,6 +1,6 @@
-# Data charts in R
+# README - Data-charts
 
-## Introduction
+## Overview
 
 This worked example is designed to show how an R script can be dockerised and could be deployed as a federated compute task with some basic characteristics that could be adapted to other use cases:
 
@@ -55,4 +55,4 @@ docker run -it\
 
 Running this as as a federated data sharing task
 
-> TODO
\ No newline at end of file
+> TODO

From 63c930c5512f87ba374cd6c056f96df8f7b5e498 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Mon, 6 Dec 2021 14:55:35 +0000
Subject: [PATCH 08/22] Updated the overview section

---
 src/data-charts/README.md | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/src/data-charts/README.md b/src/data-charts/README.md
index 20dbc40..477e101 100644
--- a/src/data-charts/README.md
+++ b/src/data-charts/README.md
@@ -2,16 +2,12 @@
 
 ## Overview
 
-This worked example is designed to show how an R script can be dockerised and could be deployed as a federated compute task with some basic characteristics that could be adapted to other use cases:
+This worked example is designed to show how an R script can be wrapped up as a docker container. To see a workflow diagram of the steps taken below go to [Containerising a script as a federated compute task](https://github.com/federated-data-sharing/common-api/blob/master/doc/User_Guide_Containerising_Tasks.md#containerising-a-script-as-a-federated-compute-task).
 
-- use a common base R docker image
-- install some dependencies
-- use a single R script as the main computation
-- configure an environment similar to the remote federated node
 
 To build up an understanding of how to run the scripts, the same task can be run locally, then via local Docker.
 
-The script itself finds any CSV files in the input folder, reads them in via `readr`, identifies numerical fields and plots histograms of each one.
+The script itself [data-charts.R](./data-charts.R) finds any CSV files in the input folder, reads them in via `readr`, identifies numerical fields and plots histograms of each one.
 
 ## Pre-requisites
 
@@ -56,3 +52,10 @@ docker run -it\
 Running this as as a federated data sharing task
 
 > TODO
+
+moved this section from overview (didn't want to delete it for now) could be deployed as a federated compute task with some basic characteristics that could be adapted to other use cases:
+
+- use a common base R docker image
+- install some dependencies
+- use a single R script as the main computation
+- configure an environment similar to the remote federated node

From 53295252cceb93cff5b32da122a944621db0d662 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Mon, 6 Dec 2021 15:02:48 +0000
Subject: [PATCH 09/22] Updated step1 section for data-charts

---
 src/data-charts/README.md | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/src/data-charts/README.md b/src/data-charts/README.md
index 477e101..3b15fee 100644
--- a/src/data-charts/README.md
+++ b/src/data-charts/README.md
@@ -13,11 +13,17 @@ The script itself [data-charts.R](./data-charts.R) finds any CSV files in the in
 
 R 3.6.1 or greater should be installed locally for the example to be run locally. The base Docker image will include a suitable version of R.
 
-## Step-by-step
+## Step 1: Run script locally on command line
 
-Create an `input` and an `output` folder here. Put one or more CSV files in the `input` folder.
+- Copy the repository on to your local machine. 
 
-Run the script directly, on the command line:
+- Create an `input` and an `output` folder here. Put one or more CSV files in the `input` folder under the directory ```.../src/data-charts```.
+
+![image](https://user-images.githubusercontent.com/91956839/144869174-6c533f6f-8772-4174-ab3a-8bbfb3279132.png)
+
+- Put one or more CSV files in the `input` folder.
+
+- Run the script directly, on the command line:
 ```sh
 rm output/*
 

From 3e1a59617411839218aad4978168b31e4246040a Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Mon, 6 Dec 2021 15:12:03 +0000
Subject: [PATCH 10/22] added step 2 and placeholder for step3

---
 src/data-charts/README.md | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/src/data-charts/README.md b/src/data-charts/README.md
index 3b15fee..26bb31a 100644
--- a/src/data-charts/README.md
+++ b/src/data-charts/README.md
@@ -36,14 +36,17 @@ Look at the output files to see the charts produced.
 
 Look at the output files to see the statistical summaries.
 
-Then, build the docker image
+## Step 2: Run containerised script via docker commandline 
+
+- Build the docker image
+
 ```sh
-docker build . -t data-profiler
+docker build . -t data-charts
 ```
 
 > Depending on your docker set up you may need to run this command prefixed by `sudo`
 
-Then run the container on the same local file:
+- Then run the container on the same local file:
 ```sh
 rm output/*
 
@@ -55,7 +58,7 @@ docker run -it\
 
 > These commands are also provided as shell scripts
 
-Running this as as a federated data sharing task
+## Step 3: Run containerised script via federated data sharing task
 
 > TODO
 

From 42eb7b83f4ab46606e049dc59b558bdf81234f81 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Tue, 7 Dec 2021 11:45:41 +0000
Subject: [PATCH 11/22] Added wording on using Rscript and OS R download.

---
 src/data-charts/README.md | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/src/data-charts/README.md b/src/data-charts/README.md
index 26bb31a..6bd1c28 100644
--- a/src/data-charts/README.md
+++ b/src/data-charts/README.md
@@ -11,7 +11,10 @@ The script itself [data-charts.R](./data-charts.R) finds any CSV files in the in
 
 ## Pre-requisites
 
-R 3.6.1 or greater should be installed locally for the example to be run locally. The base Docker image will include a suitable version of R.
+- R 3.6.1 or greater should be installed locally for the example to be run locally. Depending on the operating system (Mac, Windows, Ubuntu, etc) you are using sreach for the relevent installation staps.
+- ``Rscript`` which is a R interpreter used to execute R commands saved in a file with extesion ".R" will be needed locally. 
+- The base Docker image will include a suitable version of R.
+
 
 ## Step 1: Run script locally on command line
 

From c8c35793118972a2bac4fab4d5e1a662d1de2c49 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Tue, 7 Dec 2021 11:46:33 +0000
Subject: [PATCH 12/22] Removed comment on statistical summary

---
 src/data-charts/README.md | 2 --
 1 file changed, 2 deletions(-)

diff --git a/src/data-charts/README.md b/src/data-charts/README.md
index 6bd1c28..de425d7 100644
--- a/src/data-charts/README.md
+++ b/src/data-charts/README.md
@@ -37,8 +37,6 @@ Rscript data-charts.R
 ```
 Look at the output files to see the charts produced.
 
-Look at the output files to see the statistical summaries.
-
 ## Step 2: Run containerised script via docker commandline 
 
 - Build the docker image

From b31d85393dece26c30d396c46a0b1103f3dbf2ec Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <solmaz.eradat@aridhia.com>
Date: Tue, 7 Dec 2021 12:34:00 +0000
Subject: [PATCH 13/22] Added install package requirment for "tidyverse"

---
 src/data-charts/data-charts.R | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/data-charts/data-charts.R b/src/data-charts/data-charts.R
index 77b30b3..0a6904f 100644
--- a/src/data-charts/data-charts.R
+++ b/src/data-charts/data-charts.R
@@ -1,3 +1,5 @@
+if (!require("tidyverse")) install.packages("tidyverse")
+
 library(readr)
 library(dplyr)
 library(ggplot2)

From 9a821698f6a31a9c754187469dffaeb0d4754dfb Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Tue, 7 Dec 2021 12:41:36 +0000
Subject: [PATCH 14/22] fixed typo

---
 src/data-charts/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/data-charts/README.md b/src/data-charts/README.md
index de425d7..b49e49c 100644
--- a/src/data-charts/README.md
+++ b/src/data-charts/README.md
@@ -11,7 +11,7 @@ The script itself [data-charts.R](./data-charts.R) finds any CSV files in the in
 
 ## Pre-requisites
 
-- R 3.6.1 or greater should be installed locally for the example to be run locally. Depending on the operating system (Mac, Windows, Ubuntu, etc) you are using sreach for the relevent installation staps.
+- R 3.6.1 or greater should be installed locally for the example to be run locally. Depending on the operating system (Mac, Windows, Ubuntu, etc) you are using sreach for the relevent installation steps. 
 - ``Rscript`` which is a R interpreter used to execute R commands saved in a file with extesion ".R" will be needed locally. 
 - The base Docker image will include a suitable version of R.
 

From c91c8d0a7e2d91a79d532f8cf53154730745ebdc Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Tue, 7 Dec 2021 17:52:18 +0000
Subject: [PATCH 15/22] placeholder for README.me

---
 src/rmarkdown-report/README.md | 62 ++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)
 create mode 100644 src/rmarkdown-report/README.md

diff --git a/src/rmarkdown-report/README.md b/src/rmarkdown-report/README.md
new file mode 100644
index 0000000..8db71bf
--- /dev/null
+++ b/src/rmarkdown-report/README.md
@@ -0,0 +1,62 @@
+# README - rmarkdown-report
+
+## Overview
+
+This is a simple worked example of a summarisation task that can be wrapped up as a docker container. To see a workflow diagram of the steps taken below go to [Containerising a script as a federated compute task](https://github.com/federated-data-sharing/common-api/blob/master/doc/User_Guide_Containerising_Tasks.md#containerising-a-script-as-a-federated-compute-task).
+
+This example is intended to work on any tabular data it finds in the input folder specified. This means it can be used for data explorations: it makes few assumptions on the source data other than it being valid CSV. 
+
+The script itself [data-profiler.py](./data-profiler.py) uses the [pandas](https://pandas.pydata.org/) library to create a brief statistical profile of each field in the source data and write these to an output file.
+
+The summary result from each csv file should have the following structure: 
+![image](https://user-images.githubusercontent.com/91956839/144863924-a45cf273-6b82-49f6-bde9-9d1068d11d46.png)
+
+
+
+## Step 1: Run script locally on command line
+
+- Copy the repository on to your local machine. 
+
+- Create an `input` and an `output` folder under the directory ```.../src/data-profiler```. 
+
+![image](https://user-images.githubusercontent.com/91956839/140361598-e4eb71b2-f058-457c-9066-93022acb5e48.png)
+
+- Put one or more CSV files in the `input` folder.
+
+- Run the script directly, on the command line:
+```sh
+rm output/*
+
+export CA_INPUT_FOLDER=./input
+export CA_OUTPUT_FOLDER=./output
+
+python data-profiler.py
+```
+Look at the output files to see the statistical summaries.
+
+## Step 2: Run containerised script via docker commandline 
+
+- Build the docker image
+
+```sh
+docker build . -t data-profiler
+```
+
+> Depending on your docker set up you may need to run this command prefixed by `sudo`
+
+- Then run the container on the same local file:
+```sh
+rm output/*
+
+docker run -it\
+     --mount type=bind,source="`realpath $(pwd)/input`",target=/mnt/input\
+     --mount type=bind,source="`realpath $(pwd)/output`",target=/mnt/output\
+     data-profiler:latest
+```
+Look at the output files to see the statistical summaries.
+
+> These commands are also provided as shell scripts
+
+## Step 3: Run containerised script via federated data sharing task
+
+> TODO

From 74ff6c047fefbfb1729a5992655d8fd87787eecc Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Tue, 7 Dec 2021 17:57:25 +0000
Subject: [PATCH 16/22] replaced wording from data-chart README.md

---
 src/rmarkdown-report/README.md | 33 +++++++++++++++++++++------------
 1 file changed, 21 insertions(+), 12 deletions(-)

diff --git a/src/rmarkdown-report/README.md b/src/rmarkdown-report/README.md
index 8db71bf..e1e9d13 100644
--- a/src/rmarkdown-report/README.md
+++ b/src/rmarkdown-report/README.md
@@ -2,24 +2,27 @@
 
 ## Overview
 
-This is a simple worked example of a summarisation task that can be wrapped up as a docker container. To see a workflow diagram of the steps taken below go to [Containerising a script as a federated compute task](https://github.com/federated-data-sharing/common-api/blob/master/doc/User_Guide_Containerising_Tasks.md#containerising-a-script-as-a-federated-compute-task).
+This worked example is designed to show how an R script can be wrapped up as a docker container. To see a workflow diagram of the steps taken below go to [Containerising a script as a federated compute task](https://github.com/federated-data-sharing/common-api/blob/master/doc/User_Guide_Containerising_Tasks.md#containerising-a-script-as-a-federated-compute-task).
 
-This example is intended to work on any tabular data it finds in the input folder specified. This means it can be used for data explorations: it makes few assumptions on the source data other than it being valid CSV. 
 
-The script itself [data-profiler.py](./data-profiler.py) uses the [pandas](https://pandas.pydata.org/) library to create a brief statistical profile of each field in the source data and write these to an output file.
+To build up an understanding of how to run the scripts, the same task can be run locally, then via local Docker.
 
-The summary result from each csv file should have the following structure: 
-![image](https://user-images.githubusercontent.com/91956839/144863924-a45cf273-6b82-49f6-bde9-9d1068d11d46.png)
+The script itself [data-charts.R](./data-charts.R) finds any CSV files in the input folder, reads them in via `readr`, identifies numerical fields and plots histograms of each one.
 
+## Pre-requisites
+
+- R 3.6.1 or greater should be installed locally for the example to be run locally. Depending on the operating system (Mac, Windows, Ubuntu, etc) you are using sreach for the relevent installation steps. 
+- ``Rscript`` which is a R interpreter used to execute R commands saved in a file with extesion ".R" will be needed locally. 
+- The base Docker image will include a suitable version of R.
 
 
 ## Step 1: Run script locally on command line
 
 - Copy the repository on to your local machine. 
 
-- Create an `input` and an `output` folder under the directory ```.../src/data-profiler```. 
+- Create an `input` and an `output` folder here. Put one or more CSV files in the `input` folder under the directory ```.../src/data-charts```.
 
-![image](https://user-images.githubusercontent.com/91956839/140361598-e4eb71b2-f058-457c-9066-93022acb5e48.png)
+![image](https://user-images.githubusercontent.com/91956839/144869174-6c533f6f-8772-4174-ab3a-8bbfb3279132.png)
 
 - Put one or more CSV files in the `input` folder.
 
@@ -30,16 +33,16 @@ rm output/*
 export CA_INPUT_FOLDER=./input
 export CA_OUTPUT_FOLDER=./output
 
-python data-profiler.py
+Rscript data-charts.R
 ```
-Look at the output files to see the statistical summaries.
+Look at the output files to see the charts produced.
 
 ## Step 2: Run containerised script via docker commandline 
 
 - Build the docker image
 
 ```sh
-docker build . -t data-profiler
+docker build . -t data-charts
 ```
 
 > Depending on your docker set up you may need to run this command prefixed by `sudo`
@@ -51,12 +54,18 @@ rm output/*
 docker run -it\
      --mount type=bind,source="`realpath $(pwd)/input`",target=/mnt/input\
      --mount type=bind,source="`realpath $(pwd)/output`",target=/mnt/output\
-     data-profiler:latest
+     data-charts:latest
 ```
-Look at the output files to see the statistical summaries.
 
 > These commands are also provided as shell scripts
 
 ## Step 3: Run containerised script via federated data sharing task
 
 > TODO
+
+moved this section from overview (didn't want to delete it for now) could be deployed as a federated compute task with some basic characteristics that could be adapted to other use cases:
+
+- use a common base R docker image
+- install some dependencies
+- use a single R script as the main computation
+- configure an environment similar to the remote federated node

From 296be6ae7b9cb9126ca0bc530798d216b41176b2 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Wed, 8 Dec 2021 11:31:22 +0000
Subject: [PATCH 17/22] Updated overview section

---
 src/rmarkdown-report/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/rmarkdown-report/README.md b/src/rmarkdown-report/README.md
index e1e9d13..cbc1e3c 100644
--- a/src/rmarkdown-report/README.md
+++ b/src/rmarkdown-report/README.md
@@ -7,7 +7,7 @@ This worked example is designed to show how an R script can be wrapped up as a d
 
 To build up an understanding of how to run the scripts, the same task can be run locally, then via local Docker.
 
-The script itself [data-charts.R](./data-charts.R) finds any CSV files in the input folder, reads them in via `readr`, identifies numerical fields and plots histograms of each one.
+The script itself [generate-report.R](./generate-report.R) redners the Rmarkdown file [report.Rmd](./report.Rmd) and creates an html output with all the histogram files for the numerical variables as seen in the [data-charts example](https://github.com/solmazeradat-aridhia/common-api-examples/tree/solmazeradat-aridhia-patch-1/src/data-charts).
 
 ## Pre-requisites
 

From e27dfb7957ce12ad16851724322745d10b3a6710 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Wed, 8 Dec 2021 11:39:28 +0000
Subject: [PATCH 18/22] updated step 1

---
 src/rmarkdown-report/README.md | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/src/rmarkdown-report/README.md b/src/rmarkdown-report/README.md
index cbc1e3c..33048e2 100644
--- a/src/rmarkdown-report/README.md
+++ b/src/rmarkdown-report/README.md
@@ -14,6 +14,10 @@ The script itself [generate-report.R](./generate-report.R) redners the Rmarkdown
 - R 3.6.1 or greater should be installed locally for the example to be run locally. Depending on the operating system (Mac, Windows, Ubuntu, etc) you are using sreach for the relevent installation steps. 
 - ``Rscript`` which is a R interpreter used to execute R commands saved in a file with extesion ".R" will be needed locally. 
 - The base Docker image will include a suitable version of R.
+- ``Pandoc`` package which can be installed by running
+```
+sudo apt-get install pandoc
+```
 
 
 ## Step 1: Run script locally on command line
@@ -33,7 +37,7 @@ rm output/*
 export CA_INPUT_FOLDER=./input
 export CA_OUTPUT_FOLDER=./output
 
-Rscript data-charts.R
+Rscript generate-report.R
 ```
 Look at the output files to see the charts produced.
 

From a574f01b1076196c8d6a6e48eecfa31e7596c846 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Wed, 8 Dec 2021 11:41:24 +0000
Subject: [PATCH 19/22] updated image

---
 src/rmarkdown-report/README.md | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/rmarkdown-report/README.md b/src/rmarkdown-report/README.md
index 33048e2..ebd5d01 100644
--- a/src/rmarkdown-report/README.md
+++ b/src/rmarkdown-report/README.md
@@ -26,7 +26,8 @@ sudo apt-get install pandoc
 
 - Create an `input` and an `output` folder here. Put one or more CSV files in the `input` folder under the directory ```.../src/data-charts```.
 
-![image](https://user-images.githubusercontent.com/91956839/144869174-6c533f6f-8772-4174-ab3a-8bbfb3279132.png)
+![image](https://user-images.githubusercontent.com/91956839/145202630-1c53bf8f-de6b-4c86-b5a7-fa5555d5e7e7.png)
+
 
 - Put one or more CSV files in the `input` folder.
 

From 857ec26e1053da63ffb6310746f160946df27c4d Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Wed, 8 Dec 2021 11:44:46 +0000
Subject: [PATCH 20/22] updated step 2

---
 src/rmarkdown-report/README.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/rmarkdown-report/README.md b/src/rmarkdown-report/README.md
index ebd5d01..ecb44d2 100644
--- a/src/rmarkdown-report/README.md
+++ b/src/rmarkdown-report/README.md
@@ -47,7 +47,7 @@ Look at the output files to see the charts produced.
 - Build the docker image
 
 ```sh
-docker build . -t data-charts
+docker build . -t generate-report
 ```
 
 > Depending on your docker set up you may need to run this command prefixed by `sudo`
@@ -59,7 +59,7 @@ rm output/*
 docker run -it\
      --mount type=bind,source="`realpath $(pwd)/input`",target=/mnt/input\
      --mount type=bind,source="`realpath $(pwd)/output`",target=/mnt/output\
-     data-charts:latest
+     generate-report:latest
 ```
 
 > These commands are also provided as shell scripts

From 9c0b5ca3fbc28c62086fafc711815e26c0776a81 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Mon, 13 Dec 2021 10:34:56 +0000
Subject: [PATCH 21/22] updated file path rmarkdown-report

---
 src/rmarkdown-report/README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/rmarkdown-report/README.md b/src/rmarkdown-report/README.md
index ecb44d2..4461884 100644
--- a/src/rmarkdown-report/README.md
+++ b/src/rmarkdown-report/README.md
@@ -24,7 +24,7 @@ sudo apt-get install pandoc
 
 - Copy the repository on to your local machine. 
 
-- Create an `input` and an `output` folder here. Put one or more CSV files in the `input` folder under the directory ```.../src/data-charts```.
+- Create an `input` and an `output` folder here. Put one or more CSV files in the `input` folder under the directory ```.../src/rmarkdown-report```.
 
 ![image](https://user-images.githubusercontent.com/91956839/145202630-1c53bf8f-de6b-4c86-b5a7-fa5555d5e7e7.png)
 

From c230817133224cf998e44a33d2856544c8f55e31 Mon Sep 17 00:00:00 2001
From: Solmaz Eradat <91956839+solmazeradat-aridhia@users.noreply.github.com>
Date: Mon, 13 Dec 2021 10:40:56 +0000
Subject: [PATCH 22/22] added histogram chart

---
 src/data-charts/README.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/src/data-charts/README.md b/src/data-charts/README.md
index b49e49c..49c10b8 100644
--- a/src/data-charts/README.md
+++ b/src/data-charts/README.md
@@ -9,6 +9,10 @@ To build up an understanding of how to run the scripts, the same task can be run
 
 The script itself [data-charts.R](./data-charts.R) finds any CSV files in the input folder, reads them in via `readr`, identifies numerical fields and plots histograms of each one.
 
+A summary of the histogram charts genertated can be seen below: 
+![image](https://user-images.githubusercontent.com/91956839/145797849-b8324690-1873-409a-a30d-107ee5db96b1.png)
+
+
 ## Pre-requisites
 
 - R 3.6.1 or greater should be installed locally for the example to be run locally. Depending on the operating system (Mac, Windows, Ubuntu, etc) you are using sreach for the relevent installation steps.