Projects

This GitHub portfolio gather some of my school and work projects.
They're done with R and Phyton
website

ETL NiFi for performance-test on Dremio Iceberg

📅 SQL, BASH, PYTHON

• A study to show potentiality of Dremio Iceberg with custom data
• Built a NiFi flow to download data
• Created container with Docker for Dremio, Nessi, MinIo, NiFi
• Script for json concatenation with Python

Boolean data clustering2

🐍 PYTHON

• An evolutive study from Boolean data clustering1
• Showing capabilities of models, no deep parameters tuning
• High level theory explained in documentations
• For a more friendly read, open file word and set Navigation Pane

Boolean data clustering1

🐍 PYTHON

• A real-life project about unsupervised learning with KMeans and others
• Strictly limited by client's packages versions
• Include built-in custom functions
• Outputs saved in HDFS and followed by layers (polybase, analysis service) to end on Power Bi

Anomaly detection for ANAS

🐍 PYTHON

• Only ppt due to NDA
• It's a small POC due to a single feature analyzed (univariate)
• Tried several algorithms for unsupervised and supervised learning (supervised not shown here)

My first Data Pipeline

🐍 PYTHON

• Really simple data pipeline
• Took from a YouTube tutorial with some upgrades
• Created a presentation with PowerPoint

Data Manipulation

🐍 PYTHON

• Data Manipulation and Analysis project
• Dataset from Kaggle about billionaire companies all over the world
• Tried a presentation mode with UnicornCompanies.slides, a built-in function of jupyter notebook

Anomaly detection

📈 R

• Work project about error detections in annual reporting.
• Only R script due to NDA.

Excercises on Web Scraping

📈 R

• Self-taught excersises for work projects. In the link above there are some examples used to show to clients

Project 8: Classification

📈 R

• A tool to calculate the probability of fire in Algerian forest.
• Machine learning models used are: LDA, SVM linear and radial, KNN, random forest, rpart

Project 7: Spatial analysis

📈 R

• A study about O3 pollution in the USA (O3 pollution.pdf).
• Kriging method

Project 6: Regularized regression

📈 R

• This project is developed on three different problems:
  • Created a tool to predict a prostate cancer by using RIDGE and LASSO regressions (PROBLEMA 1);
  • Created a fake dataset to understand how RIDGE and LASSO works (PROBLEMA 2);
  • With another fake dataset is developed a Shooting Alghoritm to see how LASSO works (PROBLEMA 3)
• It's a full-immersion and deep-understanding of Ridge and Lasso alghoritm functions.

Project 5: Neural Network

🐍 PYTHON

• This was my first time with Python and my first Neural Network
• For a better view go to this link: Google COLAB
• It's everything explained there so just enjoy the reading!

Project 4: Principal component analysis

📈 R

• This is the last project for first semester Data science course. Here it's executed a really simple PCA for economics data.
• Actually I've learned new graphics and new libraries for a better visual info and anlysis.
• This project was useful to learn basic analytic skills for following cluster models (in other courses)

Project 3: Spatial regression

📈 R

• Tested the influence of neighbourhood in real estate value
• Moran test, Geary, Getis-Ord, jarque-bera..
• Spatial autoregressive model (SAM)
• Spatial lag model (SLM)
• Spatial error model (SEM)
• Spatial durbin model (SDM)
• SARAR
• Spatial durbin error model (SDEM)
• Spatial lag x (SLX)
• General nesting model (GNS)

Project 2: Logistic regression

📈 R

• Created a model to estimate a purchase by social media ads
• Dummy analysis on gender
• Confusion matrix

Project 1: Linear regression models

📈 R (it was my first time)

• Created a model to estimate revenue for movies
• Dataset got from Kaggle
• Selected the best variables for prediction and tested OLS with Jarque Bera, Breusch Pagan and Durbin Watson

That's all for now.

Bye 👋