📦 bregman-arie / devops-exercises

• A common language (for the two ends to understand)

• A way to address who you want to communicate with

• A Connection (so the content of the communication can reach the recipients)

A set of protocols that define how two or more devices can communicate with each other.

To learn more about TCP/IP, read here

Ethernet simply refers to the most common type of Local Area Network (LAN) used today. A LAN—in contrast to a WAN (Wide Area Network), which spans a larger geographical area—is a connected network of computers in a small area, like your office, college campus, or even home.

A MAC address is a unique identification number or code used to identify individual devices on the network.

Packets that are sent on the ethernet are always coming from a MAC address and sent to a MAC address. If a network adapter is receiving a packet, it is comparing the packet’s destination MAC address to the adapter’s own MAC address.

A Subnet mask is a 32-bit number that masks an IP address and divides the IP addresses into network addresses and host addresses. Subnet Mask is made by setting network bits to all "1"s and setting host bits to all "0"s. Within a given network, out of the total usable host addresses, two are always reserved for specific purposes and cannot be allocated to any host. These are the first address, which is reserved as a network address (a.k.a network ID), and the last address used for network broadcast.

Example

• Application: user end (HTTP is here)
• Presentation: establishes context between application-layer entities (Encryption is here)
• Session: establishes, manages, and terminates the connections
• Transport: transfers variable-length data sequences from a source to a destination host (TCP & UDP are here)
• Network: transfers datagrams from one network to another (IP is here)
• Data link: provides a link between two directly connected nodes (MAC is here)
• Physical: the electrical and physical spec of the data connection (Bits are here)

• Error correction - Data link

• Packets routing - Network

• Cables and electrical signals - Physical

• MAC address - Data link

• IP address - Network

• Terminate connections - Session

• 3-way handshake - Transport

Unicast: One-to-one communication where there is one sender and one receiver.

Broadcast: Sending a message to everyone in the network. The address ff:ff:ff:ff:ff:ff is used for broadcasting. Two common protocols which use broadcast are ARP and DHCP.

CSMA/CD stands for Carrier Sense Multiple Access / Collision Detection. Its primary focus is to manage access to a shared medium/bus where only one host can transmit at a given point in time.

CSMA/CD algorithm:

• Before sending a frame, it checks whether another host is already transmitting a frame.
• If no one is transmitting, it starts transmitting the frame.
• If two hosts transmit at the same time, we have a collision.
• Both hosts stop sending the frame and they send everyone a 'jam signal' notifying everyone that a collision occurred
• They are waiting for a random time before sending it again
• Once each host waited for a random time, they try to send the frame again and so the cycle starts again

• Router: a network device that connects multiple network segments together. It operates at the network layer (Layer 3) of the OSI model and uses routing protocols to direct data between networks. Routers use IP addresses to identify devices and route data packets to the correct destination.
• Switch: a network device that connects multiple devices on a LAN. It operates at the data link layer (Layer 2) of the OSI model and uses MAC addresses to identify devices and direct data packets to the correct destination. Switches allow devices on the same network to communicate with each other more efficiently and can prevent data collisions that can occur when multiple devices send data simultaneously.
• Hub: a network device that connects multiple devices through a single cable and is used to connect multiple devices without segmenting a network. However, unlike a switch, it operates at the physical layer (Layer 1) of the OSI model and simply broadcasts data packets to all devices connected to it, regardless of whether the device is the intended recipient or not. This means that data collisions can occur, and the network's efficiency can suffer as a result. Hubs are generally not used in modern network setups, as switches are more efficient and provide better network performance.

A router is a physical or virtual appliance that passes information between two or more packet-switched computer networks. A router inspects a given data packet's destination Internet Protocol address (IP address), calculates the best way for it to reach its destination, and then forwards it accordingly.

Network Address Translation (NAT) is a process in which one or more local IP addresses are translated into one or more Global IP address and vice versa in order to provide Internet access to the local hosts.

A proxy server acts as a gateway between you and the internet. It’s an intermediary server separating end users from the websites they browse.

If you’re using a proxy server, internet traffic flows through the proxy server on its way to the address you requested. The request then comes back through that same proxy server (there are exceptions to this rule), and then the proxy server forwards the data received from the website to you.

TCP 3-way handshake or three-way handshake is a process that is used in a TCP/IP network to make a connection between server and client.

A three-way handshake is primarily used to create a TCP socket connection. It works when:

• A client node sends an SYN data packet over an IP network to a server on the same or an external network. The objective of this packet is to ask/infer if the server is open for new connections.
• The target server must have open ports that can accept and initiate new connections. When the server receives the SYN packet from the client node, it responds and returns a confirmation receipt – the ACK packet or SYN/ACK packet.
• The client node receives the SYN/ACK from the server and responds with an ACK packet.

From wikipedia: "the length of time it takes for a signal to be sent plus the length of time it takes for an acknowledgment of that signal to be received"

• The client sends a Client Hello message to the server, which includes the client's version of the SSL/TLS protocol, a list of the cryptographic algorithms supported by the client, and a random value.
• The server responds with a Server Hello message, which includes the server's version of the SSL/TLS protocol, a random value, and a session ID.
• The server sends a Certificate message, which contains the server's certificate.
• The server sends a Server Hello Done message, which indicates that the server is done sending messages for the Server Hello phase.
• The client sends a Client Key Exchange message, which contains the client's public key.
• The client sends a Change Cipher Spec message, which notifies the server that the client is about to send a message encrypted with the new cipher spec.
• The client sends an Encrypted Handshake Message, which contains the pre-master secret encrypted with the server's public key.
• The server sends a Change Cipher Spec message, which notifies the client that the server is about to send a message encrypted with the new cipher spec.
• The server sends an Encrypted Handshake Message, which contains the pre-master secret encrypted with the client's public key.
• The client and server can now exchange application data.

TCP establishes a connection between the client and the server to guarantee the order of the packages, on the other hand, UDP does not establish a connection between the client and server and doesn't handle package orders. This makes UDP more lightweight than TCP and a perfect candidate for services like streaming.

ARP stands for Address Resolution Protocol. When you try to ping an IP address on your local network, say 192.168.1.1, your system has to turn the IP address 192.168.1.1 into a MAC address. This involves using ARP to resolve the address, hence its name.

• TTL (Time to Live) is a value in an IP (Internet Protocol) packet that determines how many hops or routers a packet can travel before it is discarded. Each time a packet is forwarded by a router, the TTL value is decreased by one. When the TTL value reaches zero, the packet is dropped, and an ICMP (Internet Control Message Protocol) message is sent back to the sender indicating that the packet has expired.
• TTL is used to prevent packets from circulating indefinitely in the network, which can cause congestion and degrade network performance.
• It also helps to prevent packets from being trapped in routing loops, where packets continuously travel between the same set of routers without ever reaching their destination.
• In addition, TTL can be used to help detect and prevent IP spoofing attacks, where an attacker attempts to impersonate another device on the network by using a false or fake IP address. By limiting the number of hops that a packet can travel, TTL can help prevent packets from being routed to destinations that are not legitimate.

It stands for Dynamic Host Configuration Protocol and allocates IP addresses, subnet masks, and gateways to hosts. This is how it works:

• A host upon entering a network broadcasts a message in search of a DHCP server (DHCP DISCOVER)
• An offer message is sent back by the DHCP server as a packet containing lease time, subnet mask, IP addresses, etc (DHCP OFFER)
• Depending on which offer is accepted, the client sends back a reply broadcast letting all DHCP servers know (DHCP REQUEST)
• The server sends an acknowledgment (DHCP ACK)

It is possible to have two DHCP servers on the same network, however, it is not recommended, and it is important to configure them carefully to prevent conflicts and configuration problems.

• When two DHCP servers are configured on the same network, there is a risk that both servers will assign IP addresses and other network configuration settings to the same device, which can cause conflicts and connectivity issues. Additionally, if the DHCP servers are configured with different network settings or options, devices on the network may receive conflicting or inconsistent configuration settings.
• However, in some cases, it may be necessary to have two DHCP servers on the same network, such as in large networks where one DHCP server may not be able to handle all the requests. In such cases, DHCP servers can be configured to serve different IP address ranges or different subnets, so they do not interfere with each other.

• SSL (Secure Sockets Layer) tunneling is a technique used to establish a secure, encrypted connection between two endpoints over an insecure network, such as the Internet. The SSL tunnel is created by encapsulating the traffic within an SSL connection, which provides confidentiality, integrity, and authentication.

• A client initiates an SSL connection to a server, which involves a handshake process to establish the SSL session.
• Once the SSL session is established, the client and server negotiate encryption parameters, such as the encryption algorithm and key length, then exchange digital certificates to authenticate each other.
• The client then sends traffic through the SSL tunnel to the server, which decrypts the traffic and forwards it to its destination.
• The server sends traffic back through the SSL tunnel to the client, which decrypts the traffic and forwards it to the application.

• A socket is a software endpoint that enables two-way communication between processes over a network. Sockets provide a standardized interface for network communication, allowing applications to send and receive data across a network. To view the list of open sockets on a Linux system: 

• This command displays a list of all open sockets, along with their protocol, local address, foreign address, and state.

• IPv6 (Internet Protocol version 6) is the latest version of the Internet Protocol (IP), which is used to identify and communicate with devices on a network. IPv6 addresses are 128-bit addresses and are expressed in hexadecimal notation, such as 2001:0db8:85a3:0000:0000:8a2e:0370:7334.

• Address space: IPv4 has a limited address space, which has been exhausted in many parts of the world. IPv6 provides a much larger address space, allowing for trillions of unique IP addresses.
• Security: IPv6 includes built-in support for IPsec, which provides end-to-end encryption and authentication for network traffic.
• Performance: IPv6 includes features that can help to improve network performance, such as multicast routing, which allows a single packet to be sent to multiple destinations simultaneously.
• Simplified network configuration: IPv6 includes features that can simplify network configuration, such as stateless autoconfiguration, which allows devices to automatically configure their own IPv6 addresses without the need for a DHCP server.
• Better mobility support: IPv6 includes features that can improve mobility support, such as Mobile IPv6, which allows devices to maintain their IPv6 addresses as they move between different networks.

• A VLAN (Virtual Local Area Network) is a logical network that groups together a set of devices on a physical network, regardless of their physical location. VLANs are created by configuring network switches to assign a specific VLAN ID to frames sent by devices connected to a specific port or group of ports on the switch.

• SDN stands for Software-Defined Networking. It is an approach to network management that emphasizes the centralization of network control, enabling administrators to manage network behavior through a software abstraction.
• In a traditional network, network devices such as routers, switches, and firewalls are configured and managed individually, using specialized software or command-line interfaces. In contrast, SDN separates the network control plane from the data plane, allowing administrators to manage network behavior through a centralized software controller.

• ICMP stands for Internet Control Message Protocol. It is a protocol used for diagnostic and control purposes in IP networks. It is a part of the Internet Protocol suite, operating at the network layer.

• Error reporting: ICMP messages are used to report errors that occur in the network, such as a packet that could not be delivered to its destination.
• Ping: ICMP is used to send ping messages, which are used to test whether a host or network is reachable and to measure the round-trip time for packets.
• Path MTU discovery: ICMP is used to discover the Maximum Transmission Unit (MTU) of a path, which is the largest packet size that can be transmitted without fragmentation.
• Traceroute: ICMP is used by the traceroute utility to trace the path that packets take through the network.
• Router discovery: ICMP is used to discover the routers in a network.

• SSH - 22

• SMTP - 25

• HTTP - 80

• DNS - 53

• HTTPS - 443

• FTP - 21

• SFTP - 22

Several factors can affect network performance, including:

• Bandwidth: The available bandwidth of a network connection can significantly impact its performance. Networks with limited bandwidth can experience slow data transfer rates, high latency, and poor responsiveness.
• Latency: Latency refers to the delay that occurs when data is transmitted from one point in a network to another. High latency can result in slow network performance, especially for real-time applications like video conferencing and online gaming.
• Network congestion: When too many devices are using a network at the same time, network congestion can occur, leading to slow data transfer rates and poor network performance.
• Packet loss: Packet loss occurs when packets of data are dropped during transmission. This can result in slower network speeds and lower overall network performance.
• Network topology: The physical layout of a network, including the placement of switches, routers, and other network devices, can impact network performance.
• Network protocol: Different network protocols have different performance characteristics, which can impact network performance. For example, TCP is a reliable protocol that can guarantee the delivery of data, but it can also result in slower performance due to the overhead required for error checking and retransmission.
• Network security: Security measures such as firewalls and encryption can impact network performance, especially if they require significant processing power or introduce additional latency.
• Distance: The physical distance between devices on a network can impact network performance, especially for wireless networks where signal strength and interference can affect connectivity and data transfer rates.

APIPA is a set of IP addresses that devices are allocated when the main DHCP server is not reachable

APIPA uses the IP range: 169.254.0.1 - 169.254.255.254.

OSPF (Open Shortest Path First) is a routing protocol that can be implemented on various types of routers. In general, OSPF is supported on most modern routers, including those from vendors such as Cisco, Juniper, and Huawei. The protocol is designed to work with IP-based networks, including both IPv4 and IPv6. Also, it uses a hierarchical network design, where routers are grouped into areas, with each area having its own topology map and routing table. This design helps to reduce the amount of routing information that needs to be exchanged between routers and improve network scalability.

The OSPF 4 Types of routers are:

• Internal Router

• Area Border Routers

• Autonomous Systems Boundary Routers

• Backbone Routers

• Keep caches updated (which means the request could be forwarded not to the closest data center)

Within a Spine & Leaf network there are two main tipology of switches:

• Spine Switches: Spine switches are high-performance switches arranged in a spine layer. These switches act as the core of the network and are typically interconnected with each leaf switch. Each spine switch is connected to all the leaf switches in the data center.
• Leaf Switches: Leaf switches are connected to end devices like servers, storage arrays, and other networking equipment. Each leaf switch is connected to every spine switch in the data center. This creates a non-blocking, full-mesh connectivity between leaf and spine switches, ensuring any leaf switch can communicate with any other leaf switch with maximum throughput.

• Hypertext Transfer Protocol (HTTP) - used for the webpages on the internet
• Simple Mail Transfer Protocol (SMTP) - email transmission
• Telecommunications Network - (TELNET) - terminal emulation to allow a client access to a telnet server
• File Transfer Protocol (FTP) - facilitates the transfer of files between any two machines
• Domain Name System (DNS) - domain name translation
• Dynamic Host Configuration Protocol (DHCP) - allocates IP addresses, subnet masks, and gateways to hosts
• Simple Network Management Protocol (SNMP) - gathers data on devices on the network

• Internet Protocol (IP) - assists in routing packets from one machine to another
• Internet Control Message Protocol (ICMP) - lets one know what is going such as error messages and debugging information

From the book "Operating Systems: Three Easy Pieces":

It would support the following:

• Create - allow to create new processes
• Delete - allow to remove/destroy processes
• State - allow to check the state of the process, whether it's running, stopped, waiting, etc.
• Stop - allow to stop a running process

• The OS is reading program's code and any additional relevant data
• Program's code is loaded into the memory or more specifically, into the address space of the process.
• Memory is allocated for program's stack (aka run-time stack). The stack also initialized by the OS with data like argv, argc and parameters to main()
• Memory is allocated for program's heap which is required for dynamically allocated data like the data structures linked lists and hash tables
• I/O initialization tasks are performed, like in Unix/Linux based systems, where each process has 3 file descriptors (input, output and error)
• OS is running the program, starting from main()

• Running - it's executing instructions
• Ready - it's ready to run, but for different reasons it's on hold
• Blocked - it's waiting for some operation to complete, for example I/O disk request

• I/O operations (e.g. Reading from a disk)

• Waiting for a packet from a network

The idea:

• If resources are shared between 2 or more entities (for example shared memory segments between 2 processes), the resources don't need to be copied for every entity, but rather every entity has a READ operation access permission on the shared resource. (the shared segments are marked as read-only)

• If one entity would perform a WRITE operation on a shared resource, a problem would arise, since the resource also would be permanently changed for ALL other entities sharing it.

• As a solution only, if a WRITE operation is about to be performed on a shared resource, this resource gets COPIED first and then the changes are applied.

The kernel is part of the operating system and is responsible for tasks like:

• Allocating memory

• Schedule processes

• Control CPU

Red Hat: "A hypervisor is software that creates and runs virtual machines (VMs). A hypervisor, sometimes called a virtual machine monitor (VMM), isolates the hypervisor operating system and resources from the virtual machines and enables the creation and management of those VMs."

Due to having its own drivers and a direct access to hardware components, a baremetal hypervisor will often have better performances along with stability and scalability.

Prometheus is a popular open-source systems monitoring and alerting toolkit, originally developed at SoundCloud. It is designed to collect and store time-series data, and to allow for querying and analysis of that data using a powerful query language called PromQL. Prometheus is frequently used to monitor cloud-native applications, microservices, and other modern infrastructure.

Some of the main features of Prometheus include:

• Data model: Prometheus uses a flexible data model that allows users to organize and label their time-series data in a way that makes sense for their particular use case. Labels are used to identify different dimensions of the data, such as the source of the data or the environment in which it was collected.

• Pull-based architecture: Prometheus uses a pull-based model to collect data from targets, meaning that the Prometheus server actively queries its targets for metrics data at regular intervals. This architecture is more scalable and reliable than a push-based model, which would require every target to push data to the server.

• Time-series database: Prometheus stores all of its data in a time-series database, which allows users to perform queries over time ranges and to aggregate and analyze their data in various ways. The database is optimized for write-heavy workloads, and can handle a high volume of data with low latency.

• Alerting: Prometheus includes a powerful alerting system that allows users to define rules based on their metrics data and to send alerts when certain conditions are met. Alerts can be sent via email, chat, or other channels, and can be customized to include specific details about the problem.

• Visualization: Prometheus has a built-in graphing and visualization tool, called PromDash, which allows users to create custom dashboards to monitor their systems and applications. PromDash supports a variety of graph types and visualization options, and can be customized using CSS and JavaScript.

The Prometheus architecture consists of four major components:

• Prometheus Server: The Prometheus server is responsible for collecting and storing metrics data. It has a simple built-in storage layer that allows it to store time-series data in a time-ordered database.

• Client Libraries: Prometheus provides a range of client libraries that enable applications to expose their metrics data in a format that can be ingested by the Prometheus server. These libraries are available for a range of programming languages, including Java, Python, and Go.

• Exporters: Exporters are software components that expose existing metrics from third-party systems and make them available for ingestion by the Prometheus server. Prometheus provides exporters for a range of popular technologies, including MySQL, PostgreSQL, and Apache.

• Alertmanager: The Alertmanager component is responsible for processing alerts generated by the Prometheus server. It can handle alerts from multiple sources and provides a range of features for deduplicating, grouping, and routing alerts to appropriate channels.

Compared to other monitoring solutions, such as InfluxDB, Prometheus is known for its high performance and scalability. It can handle large volumes of data and can easily be integrated with other tools in the monitoring ecosystem. InfluxDB, on the other hand, is known for its ease of use and simplicity. It has a user-friendly interface and provides easy-to-use APIs for collecting and querying data.

Another popular solution, Nagios, is a more traditional monitoring system that relies on a push-based model for collecting data. Nagios has been around for a long time and is known for its stability and reliability. However, compared to Prometheus, Nagios lacks some of the more advanced features, such as multi-dimensional data model and powerful query language.

In Prometheus, an instance refers to a single target that is being monitored. For example, a single server or service. A job is a set of instances that perform the same function, such as a set of web servers serving the same application. Jobs allow you to define and manage a group of targets together.

• Counter: A monotonically increasing value used for tracking counts of events or samples. Examples include the number of requests processed or the total number of errors encountered.

• Gauge: A value that can go up or down, such as CPU usage or memory usage. Unlike counters, gauge values can be arbitrary, meaning they can go up and down based on changes in the system being monitored.

• Histogram: A set of observations or events that are divided into buckets based on their value. Histograms help in analyzing the distribution of a metric, such as request latencies or response sizes.

• Summary: A summary is similar to a histogram, but instead of buckets, it provides a set of quantiles for the observed values. Summaries are useful for monitoring the distribution of request latencies or response sizes over time.

The exporter acts as a server, listening on a specific network port for requests from Prometheus to scrape metrics. It collects metrics from the third-party system or application and transforms them into a format that can be understood by Prometheus. The exporter then exposes these metrics to Prometheus via an HTTP endpoint, making them available for collection and analysis.

Exporters are commonly used to monitor various types of infrastructure components such as databases, web servers, and storage systems. For example, there are exporters available for monitoring popular databases such as MySQL and PostgreSQL, as well as web servers like Apache and Nginx.

• Label carefully: Careful and consistent labeling of metrics is crucial for effective querying and alerting. Labels should be clear, concise, and include all relevant information about the metric.

• Keep metrics simple: The metrics exposed by exporters should be simple and focus on a single aspect of the system being monitored. This helps avoid confusion and ensures that the metrics are easily understandable by all members of the team.

• Use alerting sparingly: While alerting is a powerful feature of Prometheus, it should be used sparingly and only for the most critical issues. Setting up too many alerts can lead to alert fatigue and result in important alerts being ignored. It is recommended to set up only the most important alerts and adjust the thresholds over time based on the actual frequency of alerts.

sum(rate(http_requests_total[1h]))

Here's an example of how to join two metrics using the join() function:

sum_series(
  join(
    on(service, instance) request_count_total,
    on(service, instance) error_count_total,
  )
)

For example, if you have a metric called httprequeststotal with a label called method, and you want to return all the values of the method label, you can use the following query:

label_values(http_requests_total, method)

100 * sum(rate(process_cpu_user_seconds_total{job="<job-name>"}[<time-period>])) by (instance) / (<time-period> * <num-cpu-cores>)

Here, is the name of the job you want to query, is the time range you want to query (e.g. 5m, 1h), and is the number of CPU cores on the machine you are querying.

For example, to get the CPU usage in percentage for the last 5 minutes for a job named my-job running on a machine with 4 CPU cores, you can use the following query:

100 * sum(rate(process_cpu_user_seconds_total{job="my-job"}[5m])) by (instance) / (5m * 4)

• Strong and static typing - the type of the variables can't be changed over time and they have to be defined at compile time

• Simplicity

• Fast compile times

• Built-in concurrency

• Garbage collected

• Platform independent

• Compile to standalone binary - anything you need to run your app will be compiled into one binary. Very useful for version management in run-time.

The result is the same, a variable with the value 2.

This should be answered based on your usage but some examples are:

• fmt - formatted I/O

Output: 2 1 3

Aritcle about sync/waitgroup

Output:
1 [5 5 5 5]
1 [5 5 5 5]
2 [5 5 5 5 5]
2 [1 2 3 4]


In mod1 a is link, and when we're using a[i], we're changing s1 value to. But in mod2, append creates new slice, and we're changing only a value, not s2.

Output: 3

MongoDB advantages are as following:

• Schemaless
• Easy to scale-out
• No complex joins
• Structure of a single object is clear

• Heterogeneous data which changes often

• Data consistency and integrity is not top priority

• Best if the database needs to scale rapidly

• A document is a record in MongoDB, which is stored in BSON (Binary JSON) format and is the basic unit of data in MongoDB.

• A collection is a group of related documents stored in a single database in MongoDB.

• An aggregator is a framework in MongoDB that performs operations on a set of data to return a single computed result.

• There is no definitive answer to which is better, it depends on the specific use case and requirements. Some explanations : Embedded documents provide atomic updates, while referenced documents allow for better normalization.

• Some ways to optimize data retrieval in MongoDB are: indexing, proper schema design, query optimization and database load balancing.

• find() returns all documents that match the query conditions.

• find_one() returns only one document that matches the query conditions (or null if no match is found).

• mongoexport
• programming languages

SQL - Best used when data integrity is crucial. SQL is typically implemented with many businesses and areas within the finance field due to it's ACID compliance.

NoSQL - Great if you need to scale things quickly. NoSQL was designed with web applications in mind, so it works great if you need to quickly spread the same information around to multiple servers

Select c.Customer_Name, o.Item
From Customers c
Left Join Orders o
On c.CustomerID = o.CustomerID;

with cat_food as (
Select CustomerID, SUM(Price) as TOTALPRICE
From Orders
Where Item like "%Cat Food%"
Group by Customer_ID
)
Select Customername, TOTALPRICE
From Customers c
Inner JOIN cat_food f
ON c.CustomerID = f.CustomerID
where c.CustomerID in (Select CustomerID from cat_food);

Although this was a simple statement, the "with" clause really shines when a complex query needs to be run on a table before joining to another. With statements are nice, because you create a pseudo temp when running your query, instead of creating a whole new table.

SELECT count(*)
FROM shawarma_purchases
WHERE
  purchased_at >= '2017-01-01' AND
  purchased_at <= '2017-31-12'

• Nova for compute resource provisioning, including VM lifecycle management.
• Neutron for networking, focusing on creating and managing networks, subnets, and routers.
• Cinder for block storage, used to attach and manage storage volumes.
• Keystone for identity services, handling authentication and authorization.

• Nova - Manage virtual instances

• Neutron - Manage networking by providing Network as a service (NaaS)

• Cinder - Block Storage

• Glance - Manage images for virtual machines and containers (search, get and register)

• Keystone - Authentication service across the cloud

• Glance - Images Service. Also used for copying or snapshot instances

• Horizon - GUI for viewing and modifying resources

• Cinder - Block Storage

• Nova - Manage virtual instances

• OpenStack is free to use - True. OpenStack is open-source software released under the Apache 2.0 license.

• The service responsible for networking is Glance - False. Neutron is the service responsible for networking. Glance is the image service.

• The purpose of tenant/project is to share resources between different projects and users of OpenStack - False. The primary purpose is to isolate resources.

• Create a Network and Subnet: First, ensure you have a private network and subnet for your instances.
• Create a Router: Create a router and connect it to the public (external) network and your private subnet.
• Launch an Instance: Launch a new instance, attaching it to your private network. It will receive a private IP address from the subnet.
• Allocate a Floating IP: Allocate a new floating IP address from the public network pool to your project.
• Associate the Floating IP: Associate the allocated floating IP with the private IP address of your instance. This allows the instance to be accessible from the internet.

• Security Group Rules: The security group attached to the instance may not have a rule allowing inbound traffic on TCP port 22 (the default SSH port).
• Firewall on the Instance: A firewall running on the instance itself (like iptables or firewalld) might be blocking the SSH port.
• SSH Service: The SSH daemon (sshd) on the instance might not be running or could be misconfigured.
• Incorrect SSH Key: You might be using the wrong private key to connect to the instance.

• Local: A local network is isolated to a single compute node and cannot be shared between multiple nodes.
• Flat: A flat network is a simple, non-VLAN-tagged network that is shared across all compute nodes.
• VLAN: A VLAN network uses 802.1q tagging to create isolated layer-2 broadcast domains.
• VXLAN: VXLAN (Virtual Extensible LAN) is an overlay network technology that encapsulates layer-2 frames in UDP packets, allowing for a large number of isolated networks.
• GRE: GRE (Generic Routing Encapsulation) is another overlay network technology that can be used to create private networks over a public network.

• Logs: Check the Cinder service logs (e.g., /var/log/cinder/cinder-volume.log, /var/log/cinder/cinder-api.log) for error messages.
• Cinder CLI: Use the cinder command-line tool to check the status of volumes, snapshots, and storage backends.
• Database: Inspect the Cinder database to check for inconsistencies in volume states or metadata.
• Backend Storage: Check the logs and status of the underlying storage system (e.g., LVM, Ceph, NFS) to identify issues with the storage itself.

• Logs: Check the Nova service logs (e.g., /var/log/nova/nova-compute.log, /var/log/nova/nova-api.log, /var/log/nova/nova-scheduler.log) for error messages.
• Nova CLI: Use the nova command-line tool to check the status of instances, hosts, and services.
• Instance Console Log: View the console log of a specific instance to see boot-up messages and other output.
• Hypervisor: Check the logs and status of the underlying hypervisor (e.g., KVM, QEMU) to identify issues with virtualization.

• DevStack: A script-based installer designed for development and testing purposes. It deploys OpenStack from the latest source code.
• Packstack: A utility that uses Puppet modules to deploy OpenStack on CentOS or RHEL. It is suitable for proof-of-concept and small-scale production environments.
• Kolla-Ansible: A set of Ansible playbooks that deploy OpenStack services as Docker containers. This method is highly scalable and recommended for production deployments.
• OpenStack-Ansible: A collection of Ansible playbooks that deploy OpenStack services directly on bare metal or virtual machines.

• Used to provision and manage virtual instances

• It supports Multi-Tenancy in different levels - logging, end-user control, auditing, etc.

• Highly scalable

• Authentication can be done using internal system or LDAP

• Supports multiple types of block storage

• Tries to be hardware and hypervisor agnostice

• nova-api - the server which serves metadata and compute APIs

• the different Nova components communicate by using a queue (Rabbitmq usually) and a database

• a request for creating an instance is inspected by nova-scheduler which determines where the instance will be created and running

• nova-compute is the component responsible for communicating with the hypervisor for creating the instance and manage its lifecycle

• One of the core component of OpenStack and a standalone project

• Neutron focused on delivering networking as a service

• With Neutron, users can set up networks in the cloud and configure and manage a variety of network services

• Neutron interacts with:

• Keystone - authorize API calls

• Nova - nova communicates with neutron to plug NICs into a network

• Horizon - supports networking entities in the dashboard and also provides topology view which includes networking details

• neutron-l3-agent - L3/NAT forwarding (provides external network access for VMs for example)

• neutron-dhcp-agent - DHCP services

• neutron-metering-agent - L3 traffic metering

• neutron-server - exposes networking API and passes requests to other plugins if required

• Management Network - used for internal communication between OpenStack components. Any IP address in this network is accessible only within the datacetner

• Guest Network - used for communication between instances/VMs

• API Network - used for services API communication. Any IP address in this network is publicly accessible

• External Network - used for public communication. Any IP address in this network is accessible by anyone on the internet

• Port

• Subnet

• Router

• Network

• L2 (Layer 2): The primary L2 component is the neutron-openvswitch-agent (or a similar agent for other plugins), which runs on each compute node and manages the local virtual switch (e.g., Open vSwitch). It is responsible for connecting instances to virtual networks and enforcing security group rules.
• L3 (Layer 3): The neutron-l3-agent is responsible for providing L3 services like routing and floating IPs. It manages virtual routers that connect private networks to external networks.

• Network: An isolated layer-2 broadcast domain.
• Subnet: A block of IP addresses that can be assigned to instances.
• Port: A connection point for attaching a single device, such as an instance, to a virtual network.
• Router: A logical entity that connects multiple layer-2 networks.
• Floating IP: A public IP address that can be associated with an instance to provide external connectivity.
• Security Group: A collection of firewall rules that control inbound and outbound traffic to instances.

• Logs: Check the Neutron service logs (e.g., /var/log/neutron/neutron-server.log, /var/log/neutron/openvswitch-agent.log, /var/log/neutron/l3-agent.log) for error messages.
• Neutron CLI: Use the neutron command-line tool to check the status of networks, subnets, ports, routers, and other networking entities.
• ip netns: Use the ip netns command to inspect network namespaces and the network configurations within them.
• ovs-vsctl and ovs-ofctl: Use these tools to inspect the configuration and flow tables of Open vSwitch bridges.
• tcpdump: Use tcpdump to capture and analyze network traffic on various interfaces to identify connectivity issues.

• Glance is the OpenStack image service

• It handles requests related to instances disks and images

• Glance also used for creating snapshots for quick instances backups

• Users can use Glance to create new images or upload existing ones

• glance-api - responsible for handling image API calls such as retrieval and storage. It consists of two APIs: 1. registry-api - responsible for internal requests 2. user API - can be accessed publicly

• glance-registry - responsible for handling image metadata requests (e.g. size, type, etc). This component is private which means it's not available publicly

• metadata definition service - API for custom metadata

• database - for storing images metadata

• image repository - for storing images. This can be a filesystem, swift object storage, HTTP, etc.

• Swift is Object Store service and is an highly available, distributed and consistent store designed for storing a lot of data

• Swift is distributing data across multiple servers while writing it to multiple disks

• One can choose to add additional servers to scale the cluster. All while swift maintaining integrity of the information and data replications.

• Container - Defines a namespace for objects.

• Account - Defines a namespace for containers

• Object - Data content (e.g. image, document, ...)

• Cinder is OpenStack Block Storage service

• It basically provides used with storage resources they can consume with other services such as Nova

• One of the most used implementations of storage supported by Cinder is LVM

• From user perspective this is transparent which means the user doesn't know where, behind the scenes, the storage is located or what type of storage is used

• cinder-api - receives API requests

• cinder-volume - manages attached block devices

• cinder-scheduler - responsible for storing volumes

• Role - A list of rights and privileges determining what a user or a project can perform

• Tenant/Project - Logical representation of a group of resources isolated from other groups of resources. It can be an account, organization, ...

• Service - An endpoint which the user can use for accessing different resources

• Endpoint - a network address which can be used to access a certain OpenStack service

• Token - Used for access resources while describing which resources can be accessed by using a scope

Using:

• Name

• ID number

• Type

• Description

• PublicURL - Publicly accessible through public internet

• InternalURL - Used for communication between services

• AdminURL - Used for administrative management

• Swift - highly available, distributed, eventually consistent object/blob store

• Sahara - Manage Hadoop Clusters

• Ironic - Bare Metal Provisioning

• Trove - Database as a service that runs on OpenStack

• Aodh - Alarms Service

• Ceilometer - Track and monitor usage

• Database as a service which runs on OpenStack - Trove

• Bare Metal Provisioning - Ironic

• Track and monitor usage - Ceilometer

• Alarms Service - Aodh

• Manage Hadoop Clusters

• Manage Hadoop Clusters - Sahara

• highly available, distributed, eventually consistent object/blob store - Swift

• You can't have OpenStack deployed without Keystone

• It Provides identity, policy and token services

• The authentication provided is for both users and services

• The authorization supported is token-based and user-based.

• There is a policy defined based on RBAC stored in a JSON file and each line in that file defines the level of access to apply

• There is a service API and admin API through which Keystone gets requests

• Keystone has four backends:

• Token Backend - Temporary Tokens for users and services

• Policy Backend - Rules management and authorization

• Identity Backend - users and groups (either standalone DB, LDAP, ...)

• Catalog Backend - Endpoints

• It has pluggable environment where you can integrate with:

• LDAP

• KVS (Key Value Store)

• SQL

• PAM

• Memcached

• Keystone gets a call/request and checks whether it's from an authorized user, using username, password and authURL

• Once confirmed, Keystone provides a token.

• A token contains a list of user's projects so there is no to authenticate every time and a token can submitted instead

• nova-api - responsible for managing requests/calls

• nova-compute - responsible for managing instance lifecycle

• nova-conductor - Mediates between nova-compute and the database so nova-compute doesn't access it directly

• nova-cert - Manages X509 certificates for secure communication.

• nova-consoleauth - Authorizes tokens for users to access instance consoles.

• nova-scheduler - Determines which compute host an instance should be launched on based on a set of filters and weights.

• Nova-novncproxy - Access through VNC connections

• Nova-spicehtml5proxy - Access through SPICE

• Nova-xvpvncproxy - Access through a VNC connection

• Django-based project focusing on providing an OpenStack dashboard and the ability to create additional customized dashboards

• You can use it to access the different OpenStack services resources - instances, images, networks, ...

• By accessing the dashboard, users can use it to list, create, remove and modify the different resources

• It's also highly customizable and you can modify or add to it based on your needs

• API is backward compatible

• There are three type of dashboards: user, system and settings

• It provides core support for all OpenStack core projects such as Neutron, Nova, etc. (out of the box, no need to install extra packages or plugins)

• Anyone can extend the dashboards and add new components

• Horizon provides templates and core classes from which one can build its own dashboard

• Puppet is a configuration management tool ensuring that all systems are configured to a desired and predictable state.

• Puppet has a primary-secondary node architecture. The clients are distributed across the network and communicate with the primary-secondary environment where Puppet modules are present. The client agent sends a certificate with its ID to the server; the server then signs that certificate and sends it back to the client. This authentication allows for secure and verifiable communication between the client and the master.

• Puppet is often compared to other configuration management tools like Chef, Ansible, SaltStack, and cfengine. The choice to use Puppet often depends on an organization's needs, such as ease of use, scalability, and community support.

• Modules - are a collection of manifests, templates, and files

• Manifests - are the actual codes for configuring the clients

• Node - allows you to assign specific configurations to specific nodes

• Facter is a standalone tool in Puppet that collects information about a system and its configuration, such as the operating system, IP addresses, memory, and network interfaces. This information can be used in Puppet manifests to make decisions about how resources should be managed, and to customize the behavior of Puppet based on the characteristics of the system. Facter is integrated into Puppet, and its facts can be used within Puppet manifests to make decisions about resource management.

• MCollective is a middleware system that integrates with Puppet to provide orchestration, remote execution, and parallel job execution capabilities.

• Hiera is a hierarchical data store in Puppet that is used to separate data from code, allowing data to be more easily separated, managed, and reused.

The Elastic Stack consists of:

• Elasticsearch

• Kibana

• Logstash

• Beats

• Elastic Hadoop

• APM Server

From the official docs:

From the official docs:

The process may vary based on the chosen architecture and the processing you may want to apply to the logs. One possible workflow is:

• The data logged by the application is picked by filebeat and sent to logstash
• Logstash process the log based on the defined filters. Once done, the output is sent to Elasticsearch
• Elasticsearch stores the document it got and the document is indexed for quick future access
• The user creates visualizations in Kibana which based on the indexed data
• The user creates a dashboard which composed out of the visualization created in the previous step

Part of a master node responsibilities:

• Track the status of all the nodes in the cluster

• Verify replicas are working and the data is available from every data node.

• No hot nodes (no data node that works much harder than other nodes)

From the official docs:

Coordinating only nodes can benefit large clusters by offloading the coordinating node role from data and master-eligible nodes. They join the cluster and receive the full cluster state, like every other node, and they use the cluster state to route requests directly to the appropriate place(s).

• Data is stored in an index
• The index is spread across the cluster using shards

Index in Elasticsearch is in most cases compared to a whole database from the SQL/NoSQL world.
You can choose to have one index to hold all the data of your app or have multiple indices where each index holds different type of your app (e.g. index for each service your app is running).

The official docs also offer a great explanation (in general, it's really good documentation, as every project should have):

From the official docs:

False. From the official docs:

• _index

• _id

• _type

• You can optimize fields for partial matching
• You can define custom formats of known fields (e.g. date)
• You can perform language-specific analysis

• If name value was different then it would update "name" to the new value
• In any case, it bumps version field by one

From the official docs:

"In the query context, a query clause answers the question “How well does this document match this query clause?” Besides deciding whether or not the document matches, the query clause also calculates a relevance score in the _score meta-field."

There are several possible answers for this question. One of them is as follows:

• Input Plugins - how to collect data from different sources

• Filter Plugins - processing data

• Output Plugins - push data to different outputs/services/platforms

According to Martin Kleppmann:

"Many processes running on many machines...only message-passing via an unreliable network with variable delays, and the system may suffer from partial failures, unreliable clocks, and process pauses."

• Network
• CPU
• Memory
• Disk

According to the CAP theorem, it's not possible for a distributed data store to provide more than two of the following at the same time:

• Availability: Every request receives a response (it doesn't has to be the most recent data)
• Consistency: Every request receives a response with the latest/most recent data
• Partition tolerance: Even if some the data is lost/dropped, the system keeps running

• The transition can take time. In other words, noticeable downtime.
• Standby server is a waste of resources - if first application server is running then the standby does nothing

Ways to improve:

• Add another load balancer
• Use DNS A record for both load balancers
• Use message queue

• The browser searches for the record of the domain name IP address in the DNS in the following order:

• Browser cache

• Operating system cache

• The DNS server configured on the user's system (can be ISP DNS, public DNS, ...)
• If it couldn't find a DNS record locally, a full DNS resolution is started.
• It connects to the server using the TCP protocol
• The browser sends an HTTP request to the server
• The server sends an HTTP response back to the browser
• The browser renders the response (e.g. HTML)
• The browser then sends subsequent requests as needed to the server to get the embedded links, javascript, images in the HTML and then steps 3 to 5 are repeated.

I like this definition from blog.christianposta.com:

From swagger.io:

False. From swagger.io:

An API gateway is like the gatekeeper that controls how different parts talk to each other and how information is exchanged between them.

The API gateway provides a single point of entry for all clients, and it can perform several tasks, including routing requests to the appropriate backend service, load balancing, security and authentication, rate limiting, caching, and monitoring.

Advantages:

• Simplifies API management: Provides a single entry point for all requests, which simplifies the management and monitoring of multiple APIs.

• Improves security: Able to implement security features like authentication, authorization, and encryption to protect the backend services from unauthorized access.

• Enhances scalability: Can handle traffic spikes and distribute requests to backend services in a way that maximizes resource utilization and improves overall system performance.

• Enables service composition: Can combine different backend services into a single API, providing more granular control over the services that clients can access.

• Facilitates integration with external systems: Can be used to expose internal services to external partners or customers, making it easier to integrate with external systems and enabling new business models.

• Authorization

• Logging

• Authentication

• Ordering

• Front-end

• Back-end

someMultiLineString: |
  look mama
  I can write a multi-line string
  I love YAML

using > will make the multi-line string to fold into a single line

someMultiLineString: >
  This is actually
  a single line
  do not let appearances fool you

They allow you reference values instead of directly writing them and it is used like this:

username: {{ my.user_name }}

Using this: --- For Examples:

document_number: 1
---
document_number: 2

• Resolve host by request to DNS resolver
• Client SYN
• Server SYN+ACK
• Client SYN
• HTTP request
• HTTP response

It consists of:

• Request line - request type

• Headers - content info like length, encoding, etc.

• Body (not always included)

• GET
• POST
• HEAD
• PUT
• DELETE
• CONNECT
• OPTIONS
• TRACE

• 1xx - informational
• 2xx - Success
• 3xx - Redirect
• 4xx - Error, client fault
• 5xx - Error, server fault

HTTPS is a secure version of the HTTP protocol used to transfer data between a web browser and a web server. It encrypts the communication using SSL/TLS encryption to ensure that the data is private and secure.

HTTP is stateless. To share state, we can use Cookies.

A proxy is a server that acts as a middleman between a client device and a destination server. It can help improve privacy, security, and performance by hiding the client's IP address, filtering content, and caching frequently accessed data.

• Proxies can be used for load balancing, distributing traffic across multiple servers to help prevent server overload and improve website or application performance. They can also be used for data analysis, as they can log requests and traffic, providing useful insights into user behavior and preferences.

A reverse proxy is a type of proxy server that sits between a client and a server, but it is used to manage traffic going in the opposite direction of a traditional forward proxy. In a forward proxy, the client sends requests to the proxy server, which then forwards them to the destination server. However, in a reverse proxy, the client sends requests to the destination server, but the requests are intercepted by the reverse proxy before they reach the server.

• They're commonly used to improve web server performance, provide high availability and fault tolerance, and enhance security by preventing direct access to the back-end server. They are often used in large-scale web applications and high-traffic websites to manage and distribute requests to multiple servers, resulting in improved scalability and reliability.

• Scalability - using a load balancer, you can possibly add more servers in the backend to handle more requests/traffic from the clients, as opposed to using one server.
• Redundancy - if one server in the backend dies, the load balancer will keep forwarding the traffic/requests to the second server so users won't even notice one of the servers in the backend is down.

• Round Robin

• Weighted Round Robin

• Least Connection

• Weighted Least Connection

• Resource Based

• Fixed Weighting

• Weighted Response Time

• Source IP Hash

• URL Hash

• A simple round robin algorithm knows nothing about the load and the spec of each server it forwards the requests to. It is possible, that multiple heavy workloads requests will get to the same server while other servers will got only lightweight requests which will result in one server doing most of the work, maybe even crashing at some point because it unable to handle all the heavy workloads requests by its own.

• Each request from the client creates a whole new session. This might be a problem for certain scenarios where you would like to perform multiple operations where the server has to know about the result of operation so basically, being sort of aware of the history it has with the client. In round robin, first request might hit server X, while second request might hit server Y and ask to continue processing the data that was processed on server X already.

Recommended read:

• Red Hat Article

• Can cause uneven load on instance (since requests routed to the same instances)

• Ensures in-proc sessions are not lost when a new request is created

• Generated by the application and/or the load balancer
• Usually allows to include custom data

• Generated by the load balancer
• Session is not sticky anymore once the duration elapsed

The maximum timeout value can be set between 1 and 3,600 seconds on both GCP and AWS.

The Creative Commons license is a set of copyright licenses that allow creators to share their work with the public while retaining some control over how it can be used. The license was developed as a response to the restrictive standards of traditional copyright laws, which limited access of creative works. Its creators to choose the terms under which their works can be shared, distributed, and used by others. They're six main types of Creative Commons licenses, each with different levels of restrictions and permissions, the six licenses are:

• Attribution (CC BY): Allows others to distribute, remix, and build upon the work, even commercially, as long as they credit the original creator.

• Attribution-ShareAlike (CC BY-SA): Allows others to remix and build upon the work, even commercially, as long as they credit the original creator and release any new creations under the same license.

• Attribution-NoDerivs (CC BY-ND): Allows others to distribute the work, even commercially, but they cannot remix or change it in any way and must credit the original creator.

• Attribution-NonCommercial (CC BY-NC): Allows others to remix and build upon the work, but they cannot use it commercially and must credit the original creator.

• Attribution-NonCommercial-ShareAlike (CC BY-NC-SA): Allows others to remix and build upon the work, but they cannot use it commercially, must credit the original creator, and must release any new creations under the same license.

• Attribution-NonCommercial-NoDerivs (CC BY-NC-ND): Allows others to download and share the work, but they cannot use it commercially, remix or change it in any way, and must credit the original creator.

A memory leak is a programming error that occurs when a program fails to release memory that is no longer needed, causing the program to consume increasing amounts of memory over time.

• File

• Block

• Object

• Data is divided to self-contained objects
• Objects can contain metadata

Pros:

• Usually with object storage, you pay for what you use as opposed to other storage types where you pay for the storage space you allocate

• Scalable storage: Object storage mostly based on a model where what you use, is what you get and you can add storage as need

• Usually performs slower than other types of storage

• No granular modification: to change an object, you have re-create it

• File Storage used for storing data in files, in a hierarchical structure
• Some of the devices for file storage: hard drive, flash drive, cloud-based file storage
• Files usually organized in directories

Pros:

• Users have full control of their own files and can run variety of operations on the files: delete, read, write and move.
• Security mechanism allows for users to have a better control at things such as file locking

A file system is a way for computers and other electronic devices to organize and store data files. It provides a structure that helps to organize data into files and directories, making it easier to find and manage information. A file system is crucial for providing a way to store and manage data in an organized manner.

Commonly used filed systems: Windows:

• NTFS

• exFAT

• HFS+

Be careful when asking this question - all companies, regardless of size, have some level of tech debt. Phrase the question in the light that all companies have the deal with this, but you want to see the current pain points they are dealing with

Similar to the tech debt question, this helps you identify any pain points with the company. Additionally, it can be a great way to show how you'd be an asset to the team.

Unit test are a software testing technique that involves systimatically breaking down a system and testing each individual part of the assembly. These tests are automated and can be run repeatedly to allow developers to catch edge case scenarios or bugs quickly while developing.

• "\d+"

• "^\w+"

• "\w+(?=\W*$)" (in most cases, depends on line formatting)

• "\b(?:\d{1,3}\ .){3}\d{1,3}\b" IPV4:(This format looks for 1 to 3 digit sequence 3 times)

• "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\ .[A-Za-z]{2,}\b"

• Resiliency: Relying on one CDN means no redundancy. With multiple CDNs you don't need to worry about your CDN being down
• Flexibility in Costs: Using one CDN enforces you to specific rates of that CDN. With multiple CDNs you can take into consideration using less expensive CDNs to deliver the content.
• Performance: With Multi-CDN there is bigger potential in choosing better locations which more close to the client asking the content
• Scale: With multiple CDNs, you can scale services to support more extreme conditions

• Scalability
• Security
• Reusability

• Complexity
• Performance overhead
• Cost and development time

• Unified tooling
• Code Sharing

• Increased complexity
• Slower cloning

• Simpler to manage
• Different teams and developers can work on different parts of the project independently, making parallel development easier.

• Code duplication
• Integration challenges

• Not suitable for frequent code changes and the ability to deploy new features
• Not designed for today's infrastructure (like public clouds)
• Scaling a team to work monolithic architecture is more challenging
• If a single component in this architecture fails, then the entire application fails.

• Each of the services individually fail without escalating into an application-wide outage.
• Each service can be developed and maintained by a separate team and this team can choose its own tools and coding language

Fault Tolerance - The ability to self-heal and return to normal capacity. Also the ability to withstand a failure and remain functional.

The load on the producers or consumers may be high which will then cause them to hang or crash.
Instead of working in "push mode", the consumers can pull tasks only when they are ready to handle them. It can be fixed by using a streaming platform like Kafka, Kinesis, etc. This platform will make sure to handle the high load/traffic and pass tasks/messages to consumers only when the ready to get them.

Caching is used to speed up read operations by storing frequently accessed data in memory or on a fast storage medium. By keeping data close to the application, caching reduces the latency and overhead of accessing data from a slower, more distant storage system such as a database or disk.

You can mention:

A common example of an embedded system is a microwave oven's digital control panel, which is managed by a microcontroller.

When committed to a certain goal, Raspberry Pi can serve as an embedded system.

There are six classes of interrupts possible:

• External
• Machine check
• I/O
• Program
• Restart
• Supervisor call (SVC)

As defined by Doug Laney:

• Volume: Extremely large volumes of data
• Velocity: Real time, batch, streams of data
• Variety: Various forms of data, structured, semi-structured and unstructured
• Veracity or Variability: Inconsistent, sometimes inaccurate, varying data

An answer from talend.com:

• Structured - data that has defined format and length (e.g. numbers, words)
• Semi-structured - Doesn't conform to a specific format but is self-describing (e.g. XML, SWIFT)
• Unstructured - does not follow a specific format (e.g. images, test messages)

• Distributed file system providing high aggregate bandwidth across the cluster.
• For a user it looks like a regular file system structure but behind the scenes it's distributed across multiple machines in a cluster
• Typical file size is TB and it can scale and supports millions of files
• It's fault tolerant which means it provides automatic recovery from faults
• It's best suited for running long batch operations rather than live analysis

HDFS Architecture

• Master-slave architecture
• Namenode - master, Datanodes - slaves
• Files split into blocks
• Blocks stored on datanodes
• Namenode controls all metadata

• Object (RGW)
• Block (RBD)
• File (CephFS)

• Reliable Autonomic Distributed Object Storage
• Provides low-level data object storage service
• Strong Consistency
• Simplifies design and implementation of higher layers (block, file, object)

• Monitor

• Central authority for authentication, data placement, policy

• Coordination point for all other cluster components

• Protect critical cluster state with Paxos
• Manager

• Aggregates real-time metrics (throughput, disk usage, etc.)

• Host for pluggable management functions

• 1 active, 1+ standby per cluster
• OSD (Object Storage Daemon)

• Stores data on an HDD or SSD

• Services client IO requests

• The client sends a request to the ceph cluster to retrieve data:

Client could be any of the following
> Ceph Block Device
> Ceph Object Gateway
> * Any third party ceph client

• The client retrieves the latest cluster map from the Ceph Monitor
• The client uses the CRUSH algorithm to map the object to a placement group. The placement group is then assigned to a OSD.
• Once the placement group and the OSD Daemon are determined, the client can retrieve the data from the appropriate OSD

• The client sends a request to the ceph cluster to retrieve data
• The client retrieves the latest cluster map from the Ceph Monitor
• The client uses the CRUSH algorithm to map the object to a placement group. The placement group is then assigned to a Ceph OSD Daemon dynamically.
• The client sends the data to the primary OSD of the determined placement group. If the data is stored in an erasure-coded pool, the primary OSD is responsible for encoding the object into data chunks and coding chunks, and distributing them to the other OSDs.

A configuration->deployment which has some advantages like:

• Deployment Speed - you configure once prior to deployment instead of configuring every time you deploy. This allows you to start instances/services much quicker.
• More immutable infrastructure - with configuration->deployment it's not likely to have very different deployments since most of the configuration is done prior to the deployment. Issues like dependencies errors are handled/discovered prior to deployment in this model.

This page explains it perfectly:

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes
MINOR version when you add functionality in a backwards compatible manner
PATCH version when you make backwards compatible bug fixes
Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.