Category Archives: Masters of Science

Protected: Wiki Diary: Learning from Data

This content is password protected. To view it please enter your password below:

Protected: Mathematics Diary, Mission A+, Eins

This content is password protected. To view it please enter your password below:

Docker to AWS, A journey

I am following this lecture to build from here: https://amigoscode.com/courses/631065/lectures/11260263

Docker Image:

Image is a template for creating an environment of your choice, snapshot, has everything to run your apps, OS, software, app code.

Below code we are pulling Apache httpd in docker desktop:

Check the images:

Run container from this image:

It will hang up so we write ctrl+c to delete it

we check with this code:

docker run

we check:

or

Docker port exposing: before that we are stopping the running container

docker stop objective_roentgen
docker stop intelligent_wiles

Now writing command for exposing the port and running the container:

d for dettach and p for port exposing t

Now the container is running perfectly and exposing the port which we can visit and check with command:

 

Now going to

will work perfectly.

It means the container port 80 is exposing to user by port 8080

Now I stopped the container:

So cannot reach to localhost:8080
Now we want to run in different port we can also follow the step:

Now we can also run multiple containers from a single command:

We stop and start again the docker container:

 

now docker help finding:

it will show like this:

removing a container from here:

we can show id  by

it will show like this:

when we need to remove running container forcefully

-f for force

Custom name for docker container:

Now stopping docker container named website

Now running again the docker container:

we check containers with command

Docker ps formatting:

 

Docker volumes: between host and controllers

pwd=present working directory

 

Kubernetes Diary

Tried this code from github issue and solved:

minikube start –no-vtx-check

Test :

kubectl get nodes

 

References:

https://learnk8s.io/installing-docker-kubernetes-windows#installing-kubernetes-on-windows-10-with-minikube

 

https://github.com/dainael/cdn_project/blob/main/kube_notes.txt

https://docs.docker.com/get-started/kube-deploy/

Latex Table and Math Equations

http://spiff.rit.edu/classes/ast601/latex/latex_table.html

https://www.latex-tutorial.com/tutorials/tables/

Protected: IOT Seminar

This content is password protected. To view it please enter your password below:

Cloud Computing Lectures and Notes Winter 2020/21

1st week:
Task: Choose a team with topic is the exercise.

You need to chose a software solution, you need make it and present.
You need to submit it before February and every week prof. needs update
It should be multi node implementation

No exam this time only project that mean

Your installation guide should provide detailed steps how to deploy,
configure and use the service
[Latex diye koire fellam]
IOT er tao kore fellam tate  duita laav hoilo
[Dui dhoroner help pawa gelo + practice]

The client-server architecture consists of 2 layers and is called two-tier model
(tier = layer)

Parental Computing is company provided the service[47 minutes in recording]

According to their areas of responsibility, 4 types of clients exist:
1 Text-/X-Terminals
2 Thin/Zero Clients
3 Applet Clients
4 Fat Clients

— My Topic For Project: Kubernetes and Docker
– Most of the cloud/DevOps engineer needs this skill and First experience in the public cloud environment (AWS or AZURE)
– good knowledge of object-oriented languages ​​(e.g. Java, Python, PowerShell or NET) and containerization strategies

References:
https://www.educative.io/blog/roadmap-to-cloud-engineer-jobs
https://www.educative.io/courses/cloud-computing-101-master-the-fundamentals

The goal of the project:
|Linux administration
Documentation reading
Architecture Building
Not so hardcore programming
If you don’t do project you will fail

 

2nd week:
We shared our topic name maybe we may change the topic name if needed.
Topic: Content Delivery Network with Kubernetes and Docker
We have 3 team members, Syed Ahmed Zaki, Daniel Von Barany, Kranthi kumar reddy Kasireddy
– We want to make a system in different server which distributes content like images, mp3, text known as content delivery networking
-We are very new to Kubernetes and Docker still researching about this
– We want to deploy in cloud maybe Minio in multi node environment or openNebula it is serverless
– It will be scalable
– Any of your suggestion Prof. ?
– My other team members and colleagues will share the details

Learned from Youtube this week about Docker and little bit Kubernetes

 

3rd week:
Notes from class lecture:
Heteregenous is better because one system and multi computer
Failover and Failback

Some multinode tutorials:
How docker works: https://aws.amazon.com/docker/

https://medium.com/@rubenafo/some-tips-to-run-a-multi-node-hadoop-in-docker-9c7012dd4e26

https://www.digitalocean.com/community/tutorials/how-to-set-up-multi-node-deployments-with-rancher-2-1-kubernetes-and-docker-machine-on-ubuntu-18-04

https://medium.com/better-programming/build-your-own-multi-node-kubernetes-cluster-with-monitoring-346a7e2ef6e2

https://github.com/rubenafo/docker-spark-cluster

 

Difference Between multi-cluster, multi-master, multi-tenant & federated Kubernetes

 

https://www.mirantis.com/blog/multi-kubernetes-kdc-quick-and-dirty-guide/

Docker book: https://docker-curriculum.com/

Awesome projects from the Docker community

 

5th Week:
Professor will take final presentation on 29 January, 2021

6th week:

I am trying to upload daniels created docker image deploying into aws ec2 server. here the tutorials i am following.

Real Time Computing Personal Note

What does real time mean ?

It means correctness and execution time of the results are guaranteed.

In non-Real time systems normally only the correctness of the result is guaranteed.

In other words: A real time system will also guaranty that a certain deadline is met.

From wikipedia professor also said:
Hard – missing a deadline is a total system failure.
Firm – infrequent deadline misses are tolerable, but may degrade the system’s quality of service. The usefulness of a result is zero after its deadline.
Soft – the usefulness of a result degrades after its deadline, thereby degrading the system’s quality of service.

Hard real-time systems are used when it is imperative that an event be reacted to within a strict deadline. Such strong guarantees are required of systems for which not reacting in a certain interval of time would cause great loss in some manner, especially damaging the surroundings physically or threatening human lives (although the strict definition is simply that missing the deadline constitutes failure of the system). Some examples of hard real-time systems:

  • car engine control system is a hard real-time system because a delayed signal may cause engine failure or damage.
  • Medical systems such as heart pacemakers. Even though a pacemaker’s task is simple, because of the potential risk to human life, medical systems like these are typically required to undergo thorough testing and certification, which in turn requires hard real-time computing in order to offer provable guarantees that a failure is unlikely or impossible.
  • Industrial process controllers, such as a machine on an assembly line. If the machine is delayed, the item on the assembly line could pass beyond the reach of the machine (leaving the product untouched), or the machine or the product could be damaged by activating the robot at the wrong time. If the failure is detected, both cases would lead to the assembly line stopping, which slows production. If the failure is not detected, a product with a defect could make it through production, or could cause damage in later steps of production.
  • Hard real-time systems are typically found interacting at a low level with physical hardware, in embedded systems. Early video game systems such as the Atari 2600 and Cinematronics vector graphics had hard real-time requirements because of the nature of the graphics and timing hardware.
  • Softmodems replace a hardware modem with software running on a computer’s CPU. The software must run every few milliseconds to generate the next audio data to be output. If that data is late, the receiving modem will lose synchronization, causing a long interruption as synchronization is reestablished or causing the connection to be lost entirely.
  • Many types of printers have hard real-time requirements, such as inkjets (the ink must be deposited at the correct time as the printhead crosses the page), laser printers (the laser must be activated at the right time as the beam scans across the rotating drum), and dot matrix and various types of line printers (the impact mechanism must be activated at the right time as the print mechanism comes into alignment with the desired output). A failure in any of these would cause either missing output or misaligned output.

In the context of multitasking systems the scheduling policy is normally priority driven (pre-emptive schedulers). In some situations, these can guarantee hard real-time performance (for instance if the set of tasks and their priorities is known in advance). There are other hard real-time schedulers such as rate-monotonic which is not common in general-purpose systems, as it requires additional information in order to schedule a task: namely a bound or worst-case estimate for how long the task must execute. Specific algorithms for scheduling such hard real-time tasks exist, such as earliest deadline first, which, ignoring the overhead of context switching, is sufficient for system loads of less than 100%.[6] New overlay scheduling systems, such as an adaptive partition scheduler assist in managing large systems with a mixture of hard real-time and non real-time applications.

Firm real-time systems are more nebulously defined, and some classifications do not include them, distinguishing only hard and soft real-time systems. Some examples of firm real-time systems:

  • The assembly line machine described earlier as hard real-time could instead be considered firm real-time. A missed deadline still causes an error which needs to be dealt with: there might be machinery to mark a part as bad or eject it from the assembly line, or the assembly line could be stopped so an operator can correct the problem. However, as long as these errors are infrequent, they may be tolerated.

Soft real-time systems are typically used to solve issues of concurrent access and the need to keep a number of connected systems up-to-date through changing situations. Some examples of soft real-time systems:

  • Software that maintains and updates the flight plans for commercial airliners. The flight plans must be kept reasonably current, but they can operate with the latency of a few seconds.
  • Live audio-video systems are also usually soft real-time. A frame of audio that’s played late may cause a brief audio glitch (and may cause all subsequent audio to be delayed correspondingly, causing a perception that the audio is being played slower than normal), but this may be better than the alternatives of continuing to play silence, static, a previous audio frame, or estimated data. A frame of video that’s delayed typically causes even less disruption for viewers. The system can continue to operate and also recover in the future using workload prediction and reconfiguration methodologies.[7]
  • Similarly, video games are often soft real-time, particularly as they try to meet a target frame rate. As the next image cannot be computed in advance, since it depends on inputs from the player, only a short time is available to perform all the computing needed to generate a frame of video before that frame must be displayed. If the deadline is missed, the game can continue at a lower frame rate; depending on the game, this may only affect its graphics (while the gameplay continues at normal speed), or the gameplay itself may be slowed down (which was common on older third- and fourth-generation consoles

WCET(Worst Case Execution Time):
It must not be less than the real WCET (otherwise the
deadline will be execced sometime).
2 It should be as near as possible to the real WCET (otherwise
computing power will be wasted −→ the system gets more
expansive)

Two methods:
Statical WCET Analysis: Calculating execution time
based on the computer program.
Dynamical WCET Analysis: Measurement of an
adequate number of execution runs on a certain
hardware

 

Reference: https://www.geeksforgeeks.org/real-time-systems/
https://en.wikipedia.org/wiki/Real-time_computing

Pattern Oriented Software Engineering Course

Here I will put all the class lectures and exercises of pattern oriented software architectures.

1st class 12.11.2020 :
Strategt pattern and it’s UML diagram and when in details for example duck

Now from Exercise Sheet -1 :

ddfd

Java learned as so far:

Java variables,
Array, 2d array
Method
Different method invoking and return type
Class
– Class is actually a blueprint/specification/design
– A class can create as many as objects it needs to create
Object
– Earth is a software application where human and other thing is software objects.For creating object we need to create a human class
– Objects do particular behavior in a class
– Object exist only in application runtime actually when application is running
– Objects have identity,state and behavior

this keyword – it represent current object
Instance
Constructor – The costructor is a special method with the same name as the class and it’s used to instantiate objects from that class
– Every class has a by default constructor if we don’t put it manually
– The constructor is special method containing instructions for object creation. it can be said it is so called birth method. It has instructions for how the object will be born when the application starts up
Static keyword
Stack and heap memory
Garbage collection
Object oriented Programming = better organizing of code, it is a way to organize our code from multiple files by creating objects
Local variable
Reference variable
Instance variable
Inheritance
– Inheritance is the approach to get/transfer behaviour of one class to another
– We use extends keyword for inheriting, in inheritance we inherit from parent class which known as base class and the class which needs inheritation is known as child class or sub class/derived class
super keyword it extract constructor and it’s variable from parent class but reverse is not possible
Overrides
– Same method name with different purpose
– Same method name in parent class but in child class the method name is same but with different purpose
– Overrides means replace
– Override is not a good idea sometimes so we go for interface
Interfaces
– Interface is a contract with a class. The class needs to implements that method of the interface according to contract
– Interface has a method which does not need body, so it is just only to define, it is called as abstract method
– one parent class
– We use implements keyword while putting interface in a class
Abstract class
– When you don’t need all methods to invoke you can use abstract method, you can not make object from abstract class. Abstract method needs to be in abstract class. We use abstract keyword for both method and object to define abstract class and methods
– you can only extend abstract class but can not instantiate abstract class
– We can define abstract class as a type while creating object
Polymorphism

Loops
Exception Handling

Collection Framework
Wrapper class
Threading
Concurrency
Database

Jshell
Functional Interfaces
Lambda expressions

-Lambda helps us to use separate from associated object
– it can run without class
– We need to use lot of interfaces for this

Streams

Ei bold kora jayga diye project banaye banaye clear korte hobe
Italic kora jaygagula motamuti clear asey In Sha Allah, Alhamdulillah
The source is : Imtiaz Ahmed’s complete Java course

 

Master Object Oriented Design in Java course note

Association: It defines relationship between classes, it defines how the software will behave
Dependency Association: For example from code:

A driver receives a vehicle reference only as driver then only he can drive or accelerate

Composition: A particular object . It imply ownership
for example in this code:

Aggregation Association: It does not imply ownership

Here Department to SpanishCourse relation is composite

But Course to Student relationship is not composite because it does not imply ownership a student can take many courses or not

We find this relationship from above diagram, the block diamond and white diamond is the part of UML diagram

Forming Association between objects:

has a relationship in customer .

Now when to stop delivery

Overview of software design:
For example: Hospital Employee Management System
Task of the software:
Hire and Terminate Employees
Print Reports(XML,CSV Formats)

Problem Statement:
Current status of the software

Tips for design:
– You must draw design in paper or whiteboard don’t go to code directly
– Don’t overdesign or overdraw
– We have to develop iteratively but early steps should be kept simple
– Class names should be nouns based on their intention

Single Responsibility Priniciple(SRP): Not all the things in the same class

Don’t repeat yourself(DRY):

Arrow sign always describe that it is depends on that dependency class:

 


DAO=Data Access Object

Single Responsibility Principle Special Notes:
A class should have only one single reason to change

 

Open closed principle sayings about class design:

Classes should be open for extension but closed for modification.

Open Closed Principle and Strategy pattern:
Software modules should be open for extension and closed for modification
In development lifecycle software requirements needs to be constantly pour in. If developed need to change the already made codes the design is fragile. Fragile means broken.

So open closed principle save engineers from fragile design

instanceof operator we used here in the code:
HotelManagementSystem class:

Here it is like a junk drawer

So we will make it changes because it is violating OCP principle. This chamging process is known as Strategy pattern. So let’s do it.

We could also do it with interface class except abstract class

Example after applying Strategy pattern:
Nurse.java

Doctor,java

HospitalManagement.java

 

Employee.java

 

EmergencyRoomProcess.java

To String Tutorial:

DemoToString.java

Dependency Inversion Principle:
Device Manufacturing Company has:

The manufacturing process:
1. Assembly
2. Testing
3. Packaging
4. Storage

GeneralMAnufacturingProcess
UML diagram:

We depend on something in driving like car is stopping by traffic light
Imagine a life every car has different mechanism
Violation of Dependency Inversion Principle
When higher level modules depend on lower level ones succeptible to change
Dangers of coupling:
Tight coupling :

Well design software has single responsibility
This method says high level modules should not depend on lowlevel modules. It should depend on abstractions
Abstract classes and Interfaces don’t change as often as concrete classes/ derivatives
Cohesion means related things together.

This code I tried to run in class inside class but did not worked. Then I downloaded course code and run worked then again I tried to do with my directory structure then it worked.

Quizes from OOP course:
Q1:
Q2: The Dependency Inversion Principle states
Interfaces and abstract classes change far less often the concrete derivatives. Concrete classes should depend on Interfaces and abstract classes because they change less often and are higher level modules

Q3: In general the dependency inversion principle states that code should only depend on things that don’t change often
True
Liskov Substitution Principle:
Subtypes must be substitutable for their basetypes

There are some patterns which is not good known as Anti-pattern:
Here the link can be found


This is a ISA relationship, Nurse and Doctor depends on Employee class. Employee class is Abstract class

When it is adding a substitute teacher who does not work as like a teacher it violates the LISKOV pronciple as per this line of definition: Subtypes must be substitutable for their basetypes

So we changed to a new UML which does not violate LISKOV principle and did the code.

New UML:


Question 1:
The Liskov Substitution principle states, that child classes should not be substitutable in place of their parents.

Interface Segregation Principle:

Fat class is a class which has too many responsibilities.

CustomerTransaction

From Quiz: Main Kotha about Interface Segregation Principle:

This principle does not have to do with the number of dependencies that may exist in an application. The principle simply states that modules should not have dependencies on code they do not use.

If a module does not use functionality from another module, there is no reason to have a direct dependency between them. There should be an abstraction in between to segregate the 2 modules

Dependency Injection: 
When the application is not running there is no object because onbject only available in runtime, you will find only some classes

Tight Coupling:
Cohesion:
A very popular dependency injection implementation is spring framework bundled with spring container.

Dependency injection is

Dependency Injection Using Spring:
Observer Pattern:
The observer pattern is a software design pattern in which an object, called the subject, maintains a list of its dependents, called observers, and notifies them automatically of any state changes, usually by calling one of their methods.

One basic UML diagram from wikipedia:

Created one for the course:

Builder Pattern:
When there is so many constructors and tough to handle it. Example codes with some bugs. Will do the solution later.

code example:
user.java

app.java

Factory Pattern:

UML Diagram:

codes:
Vehicle.java

VehicleFactory.java

App.java

VehicleType.java

ElectricCar.java

Car.java

Truck.java

Singleton Design Pattern: It only uses once in an application.

PerformanceStage

Application.java

 

 

Course Source Imtiaz Ahmed Master OOP Design udemy course
Best Book Rereference/Website reference for learning Java: https://beginnersbook.com/2013/12/java-string-equals-and-equalsignorecase-methods-example/

Protected: Introductory Data Analysis: winter 2020 by Andersson

This content is password protected. To view it please enter your password below:

Linear Regression: Influential Observations

https://www.youtube.com/watch?v=fJSXS4oVf88

Confusion matrix

https://www.dataschool.io/simple-guide-to-confusion-matrix-terminology/#:~:text=A%20confusion%20matrix%20is%20a,the%20true%20values%20are%20known.&text=The%20classifier%20made%20a%20total,the%20presence%20of%20that%20disease).

https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62

Hypothesis Testing

Outliers in DM