Skip to content

Doc - CliSK development example - #78

Open
KillianCourvoisier wants to merge 8 commits into
masterfrom
doc/DevelopmentExample
Open

Doc - CliSK development example#78
KillianCourvoisier wants to merge 8 commits into
masterfrom
doc/DevelopmentExample

Conversation

@KillianCourvoisier

@KillianCourvoisier KillianCourvoisier commented Apr 23, 2025

Copy link
Copy Markdown
Contributor

🔴⚠️ This PR is an example for CliSK development . DO NOT MERGE ⚠️🔴

@KillianCourvoisier
KillianCourvoisier force-pushed the doc/DevelopmentExample branch 3 times, most recently from df9157d to 15d9e56 Compare May 6, 2025 16:09
@KillianCourvoisier
Adapt manifest
db9fede 
First of all you will need to setup the konnector's manifest and adapt it to the targeted website.
Change all needed values where you see fit such as name and slug of the konnector, your name and link in the developer object and all the needed descriptions, category etc ... However, some of the manifest values must not be touched like the version or the permissions.
Think that if you don't really know what to put in replacement, don't touch it, we will adjust if needed when we will review your work.

missed manifest infos

add spaces in manifest name
@KillianCourvoisier
Clean template's code
de06976 
We begin with the cleaning of the template's code to make place for your own.
You can see we're keeping some mandatory functions. They will be overriden as they will be launched by the pilot as the path to follow during konnector's execution.
We're also renaming the Class to fit targeted website.
@KillianCourvoisier
EnsureNotAuthenticated
74124b6 
In this commit we are starting by ensuring there is no active session on the targeted website so we can begin on a fresh start. As it is (and will be until we create a cozy account and saved some credentials) the first ever konnector's run on the device.
To cover for the automated logout too, we reach the "login successfully" page of the targeted website.
When one or the other element is detected, we're launching `checkAuthenticated` in the worker to determine if the logout button is visible on the page.
If it is not, here we go, we are ready to let the user login ! If it is, we proceed to click on the logout button and wait for the wanted form input element to show up, indicating the logout has been done properly.
@KillianCourvoisier
EnsureAuthenticated
412c697 
Now we are absolutely sure there is no active session on the targeted website, we can show the webview to the user and let him login onto the website for us.
For the moment, this is all this function will do, but we will come back on it when we have saved some credentials to use.
@KillianCourvoisier
Credentials Interception
1a6e710 
Here we implement the credentials interception to be able to save them on the devicea little bit later. For this, you need those two worker's functions `onWorkerReady` and `onWorkerEvent`. First we need to tell the pilot we're gonna listen to the worker event, the line is added at the beginning of `ensureAuthenticated` to get the listener ready right at the start.
Then we create a listener on the wanted event. Here we're using the click on the form's submit button. Like this, when the user click on the button, the event is catched and we can scrape the inputs values on the spot, before the page load. If the user used a wrong combination, he will need to click the button again, replacing the first interception by the new.
After that the worker emits an event containing the event name, and its payload (an object containing the credentials). It is catched by the pilot, checked, and the credentials are placed in the store.
This type of interceptions can be done for various other reasons like detecting an error, wait for a DOM modification (like watching a loading spinner finish before taking action)  etc ... it can be very useful.
@KillianCourvoisier
GetUserDataFromWebsite
56ae90d 
In this function, we'll need to find a `sourceAccountIdentifier` to return, so we can create an account and a trigger in the cozy instance. It will also be used to name the destination directory.
If the credentials interception went well, we supposed to have the user's login credential in the store but we're gonna verify this. First check if we found some saved credentials from previous run, so it's already ready for later runs. Then check in the store. We will prefer the intercepted credentials as it is what the user had just typed, he may have changed his login username recently. This way we're ensured to have the most recent credentials every time. Alternatively, the user may sign in with his email, but his profile may show his username or ID instead. If there are no successful interceptions and no saved credentials, we may need to scrape it on the website.
In this case, we meet our first handmade `worker` function !
Named `findValidSAI` (for good mesure, but not mandatory), we're gonna launch it by giving it as the argument to `runInWorker` `pilot's` method.

/!\ It is very important to add every handmade `worker` functions to the `additionalExposedMethodsNames` array at the end of the file, otherwise it will not be "seen" by the `pilot` and will never be executed /!\

In this function we're gonna scrape on the website the needed data. Here for example we're finding the element containing the username, splitting it's content as it is part of a larger sentence and extract "student" as the identifier we want. Note that you may also find an identifier in numerous different ways, it could be intercepted in a response, found in the local/session storage, or hidden in a burger menu so stay creative.
@KillianCourvoisier
KillianCourvoisier force-pushed the doc/DevelopmentExample branch from a529f12 to 56ae90d Compare May 6, 2025 16:12
@KillianCourvoisier
Fetch
0d4f8f7 
Time to find the sought files and save them in the cozy instance !
Right before the fun begins, we will save the intercepted credentials if any, that way we have them for future execution, and possibly make the login automated so we didn't even need the user to do anything.
Following that, we will navigate to the first page containing the files if needed. We'll find the number of pages we have to scrape (for this example we fixed the limit to 3 pages to avoid scraping the whole 50 of them). With this number, we'll enter a loop to get all the files needed on every pages. Once in the loop, we'll use a handmade `worker`'s function to get and format all the files of the current page. This function will return the formatted files to the pilot for it to store them waiting for the loop to complete. After that, we navigate to the next page and repeat the process until the condition to end the loop is met. Right after, we pass the fullfilled array of files to the `saveFiles` function for it to save them on the cozy instance.
@KillianCourvoisier
Identity
583b979 
Now all the fetching has been done, the last thing to do (when possible) is to fetch the user identity. As the website used for this example does not contain any user's account page, we will just show how to build a userIdentity to be saved on a cozy. If needed, navigate to the info page of the user, then just scrap everything usefull for an identity. Names, address, email and phone number are the common data you can find, but depending of the website, you might want to save a social security number or anything else specific from the targeted website. The structure is mandatory, the email must be a string as an array entry, the address and phone must be in an object as an array's entry as well. Sometimes, splitting an address in streets/postcode/city might not be easy, it can only be the "formattedAddress". No keys are specifically mandatory, it will depends on what you can find on the website. But if you find some, the structure is mandatory or it will not be displayed correctly in the cozy instance.
And with that, we're all done, a brand new CliSK can be added to the list !
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@KillianCourvoisier