MODULES

Portfolio Project :-Building a Free Tender Announcement Website

# Building a Free Tender Announcement Website

This past year, I embarked on a project that aimed to address a common challenge: finding timely tender announcements. As someone who frequently seeks out tender opportunities, I noticed that existing websites often fell short in providing up-to-date information. To tackle this issue, I decided to create my own website—a platform that would aggregate and organize tender announcements from various sources.

## The Problem

The problem was twofold: first, existing websites lacked real-time updates, leaving me to manually check multiple platforms daily. Second, the data from these websites was often disorganized and required human intervention to process effectively.

## Our Solution

During my ALX DevOps training, I teamed up with Natnael Gedlu to build our **Free.Tender** website. Our goal was clear: create a user-friendly webpage that autonomously fetched data from different sources and presented it in an organized format.

### Front-End and Back-End Focus

Natnael took charge of the front-end design, ensuring that our website was intuitive and visually appealing. Meanwhile, I focused on the back-end development, where the real technical challenges awaited.


### Technical Challenges

The most difficult aspect of our project was fetching data from external websites using the `curl` tool. We needed to extract relevant information from HTML code and transform it into a structured format for our database. Here's how we tackled it:


1. **Using `curl` for Data Retrieval**: We leveraged `curl` to make HTTP requests to various tender announcement websites. By specifying the `-X GET` option, we ensured that we were making GET requests to retrieve data.


2. **Parsing HTML Data**: Once we received the HTML content, we needed to extract relevant details such as tender descriptions, deadlines, and contact information. Parsing HTML is notoriously tricky due to its unstructured nature. We wrote custom functions (using `CURLOPT_WRITEFUNCTION`) that processed the raw HTML data and extracted the necessary fields.


3. **Data Organization**: The extracted data needed further processing before being stored in our database. We sorted tenders by category, region and languages classification.

# Summarize

We use a nice template which we found it from niceadmin website.

What happens when you type https://www.google.com in your browser and press Enter?

You might think that typing https://www.google.com in your browser and pressing Enter is a simple action that takes you to the Google homepage. But behind the scenes, there is a lot more going on than meets the eye. In this blog post, we will explain the steps involved in this process and how they affect your browsing experience.

Step 1: Parsing the URL

The first thing that happens when you type https://www.google.com in your browser is that the browser parses the URL (Uniform Resource Locator) to extract its components. A URL consists of several parts, such as the protocol, the domain name, the path, and the query string. For example, in https://www.google.com/search?q=alx+africa, the protocol is https, the domain name is www.google.com, the path is /search, and the query string is q = 'alx africa’.

The protocol specifies how the browser should communicate with the server that hosts the website. The most common protocols are http (Hypertext Transfer Protocol) and https (Hypertext Transfer Protocol Secure), which use port 80 and port 443 respectively. The https protocol encrypts the data exchanged between the browser and the server, making it more secure and preventing eavesdropping or tampering.

The domain name identifies the server that hosts the website. The domain name is composed of one or more labels separated by dots, such as www.google.com. Each label can have up to 63 characters, and the whole domain name can have up to 253 characters. The rightmost label is called the top-level domain (TLD), such as .com, .org, .net, etc. The TLD indicates the type or category of the website, such as commercial, organizational, network, etc. The second-level domain (SLD) is the label to the left of the TLD, such as google. The SLD usually represents the name of the organization or entity that owns or operates the website. The third-level domain (TLD) is the label to the left of the SLD, such as www. The TLD usually indicates a specific subdomain or service within the website, such as web, mail, ftp, etc.

The path specifies the location of a specific resource within the website, such as a web page, an image, a video, etc. The path consists of one or more segments separated by slashes, such as /search. Each segment can have up to 255 characters, and can contain alphanumeric characters, hyphens, underscores, periods, and other symbols. The path can also be empty, meaning that the default resource of the website is requested.

The query string provides additional information or parameters to the server about the request. The query string consists of one or more key-value pairs separated by ampersands (&), such as q=bing. Each key-value pair can have up to 2048 characters, and can contain alphanumeric characters, hyphens, underscores, periods, and other symbols. The query string can also be empty, meaning that no parameters are provided.

Step 2: Resolving the domain name

The next thing that happens when you type https://www.google.com in your browser is that Resolving the domain name. This is a useful skill for web developers and network administrators who want to troubleshoot DNS issues or test their website's availability. in our example demo when you type https://www.google.com. This is a popular search engine that can also perform DNS lookups. In the search box, type "nslookup example" and hit enter. You will see a result like this:

Server: 8.8.8.8
Address: 8.8.8.8#53

Non-authoritative answer:
Name: example
Address: 93.184.216.34

The server and address lines show the IP address of the Google DNS server that performed the lookup. The non-authoritative answer line shows the domain name and the IP address of the website you are looking for. In this case, example has an IP address of 93.184.216.34.

You can also use other options with nslookup, such as -type=A to specify the record type, or -debug to see more details about the query and response. For more information, you can check the nslookup documentation or type "nslookup -help" in the search box.

Step 3: HTTPS/SSL

using our example https://www.google.com website as an . HTTPS stands for Hypertext Transfer Protocol Secure, and it is a protocol that encrypts the data sent between a client and a server using SSL (Secure Sockets Layer) certificates. SSL certificates are digital documents that verify the identity and authenticity of a website, and they are issued by trusted authorities called Certificate Authorities (CAs).

To use HTTPS/SSL, you need to have a valid SSL certificate for your website, and you need to configure your web server to listen on port 443, which is the default port for HTTPS. You also need to redirect any HTTP requests (port 80) to HTTPS, so that your users always access your website securely. There are different ways to do this depending on your web server software, such as Apache, Nginx, or IIS.

To demonstrate how HTTPS/SSL works, let's use https://www.google.com as an example. when you open your web browser and type https://www.google.com in the address bar, you will see a padlock icon next to the URL, indicating that the connection is secure. If you click on the padlock, you will see more details about the SSL certificate that Google uses, such as the issuer, the validity period, and the encryption algorithm. You can also view the certificate itself by clicking on "Certificate".

The certificate shows that Google's website is verified by Google Trust Services LLC, which is a CA that Google owns and operates. The certificate also shows that Google uses a strong encryption algorithm called RSA with 2048-bit key length, which means that it is very difficult for anyone to intercept or tamper with the data sent between your browser and Google's server.

By using HTTPS/SSL, you can ensure that your web requests are secure and private, and that you are communicating with the legitimate website that you intended to visit. HTTPS/SSL also helps protect your website from malicious attacks such as phishing, man-in-the-middle, or spoofing. Therefore, it is highly recommended that you use HTTPS/SSL for your website, especially if you handle sensitive information such as personal data, passwords, or payment details.

Step 4 : Load-Balancer

In this blog post section, I will demonistrate how to set up a simple load-balancer using https://www.google.com as an example. A load-balancer is a device or software that distributes incoming network traffic across multiple servers, improving the performance and availability of your web applications.

To create a load-balancer, you will need the following:

- A domain name that points to your load-balancer's IP address. For this example, I will use google.com.
- Two or more servers that host your web application. For this example, I will use server1.google.com and server2.google.com.
- A load-balancing software that runs on your load-balancer. For this example, I will use nginx, a popular open-source web server and reverse proxy.

The steps to configure the load-balancer are as follows:

1. Install nginx on your load-balancer. You can follow the official documentation for your operating system here: https://nginx.org/en/docs/install.html
2. Edit the nginx configuration file, usually located at /etc/nginx/nginx.conf, and add the following lines:

http {
        upstream backend {
            server server1.google.com;
            server server2.google.com;
        }

        server {
            listen 80;
            server_name google.com;

            location / {
                proxy_pass http://backend;
            }
        }
    }

This tells nginx to create a group of servers called backend, consisting of server1.google.com and server2.google.com, and to forward all requests for google.com to one of them.

3. Restart nginx to apply the changes. You can use the command `sudo service nginx restart` or `sudo systemctl restart nginx`, depending on your operating system.
4. Test your load-balancer by visiting http://google.com in your browser. You should see the same web page as if you visited https://www.google.com directly. You can also check which server is handling your request by inspecting the HTTP headers using a tool like curl or Postman. You should see a header like `X-Server: server1.google.com` or `X-Server: server2.google.com`, indicating which server nginx chose for your request.

Congratulations! You have successfully set up a load-balancer using https://www.google.com as an example. You can now scale your web application by adding more servers to the backend group and enjoy the benefits of load-balancing.

Step 5: Web Server

In this blog section, I will Demonistrate to you how to create a simple web server using https://www.google.com as the base URL. A web server is a program that listens for incoming requests from clients (such as web browsers) and sends back responses (such as HTML pages). To create a web server, we need two things: a URL and a handler function.

A URL is a string that identifies a resource on the web, such as https://www.google.com. A handler function is a function that takes a request object and returns a response object. The request object contains information about the client's request, such as the method, the path, the headers, and the body. The response object contains information about the server's response, such as the status code, the headers, and the body.

To create a web server using https://www.google.com as the base URL, we can use the built-in module `http` in Node.js. Node.js is a JavaScript runtime environment that allows us to run JavaScript code outside of a browser. The `http` module provides methods for creating and managing HTTP servers and clients.

First, we need to import the `http` module using the `require` function:

Next, we need to create a server object using the `http.createServer` method. This method takes a handler function as an argument and returns a server object. The handler function will be called whenever the server receives a request.

 Step 6 : Application Server
In this blog Section, I will try in short, you how to create a simple application server using https://www.google.com as the backend. This is a useful example for learning how to use web services and APIs in your own projects.

An application server is a special kind of server that can host, install, and operate applications. It can handle different types of applications, from simple to complex, and from small to enterprise-level. An application server also provides a platform for the application logic, which is the core functionality of any application.
An application server has many advantages for application development and deployment. Some of these are:
- It reduces the size and complexity of client programs, which makes them easier to develop and maintain.
- It improves the performance of applications by caching and controlling the data flow between the server and the clients.
- It enhances the security of applications by implementing encryption, authentication, and authorization for data and user traffic.

The first step is to install the required packages for your programming language of choice. For this example, I will use Python and the requests library. You can install them using pip:

pip install requests

The next step is to write a function that takes a query as an input and returns the result from https://www.google.com. For example, if the query is "python tutorial", the function should return the HTML content of the first page of Google search results for that query. Here is the code:

import requests

def google_search(query):
  url = "https://www.google.com/search?q=" + query
  response = requests.get(url)
  return response.text

The final step is to create a server that listens for requests from clients and calls the google_search function with the query parameter. For this example, I will use Flask, a lightweight web framework for Python. You can install it using pip:

pip install flask

Here is the code for the server:

from flask import Flask, request
app = Flask(__name__)

@app.route("/")
def index():
  query = request.args.get("query")
  if query:
    result = google_search(query)
    return result
  else:
    return "Please provide a query parameter"

if __name__ == "__main__":
  app.run()


Step 7 : Database

Databases are essential tools for storing and managing data in a logical and organized way. They allow us to access, modify, and analyze data efficiently and securely. Databases are widely used in various domains, such as business, education, healthcare, and more.
The primary role of a database is to provide a mechanism for storing and retrieving data in an efficient and secure manner. Databases can store different kinds of data, such as customer data, financial records, inventory data, and more. They can also handle complex data relationships and queries.
Another important function of a database is to ensure data security and integrity. This means that databases have features that control who can access and modify the data, how to backup and restore the data in case of failure, and how to maintain data consistency and accuracy across different applications.

Databases are powerful tools that can help us store and manage data effectively and securely. They are used in many applications and scenarios that require data storage and manipulation.

One example of a database server is Google's Bigtable, which is a distributed storage system for managing structured data. Bigtable powers many Google products, such as Google Search, Gmail, Google Maps, and YouTube. Bigtable stores data in tables that can have billions of rows and millions of columns, and can scale to thousands of machines across multiple data centers. Bigtable supports fast and flexible data access, such as random reads and writes, range queries, and map-reduce operations.


I hope this blog post was helpful and informative. If you have any questions or feedback, please leave a comment below.

ACCOUNTING

ACCOUNTING

Manage your finances with the 100% open source accounting software

EDUCATION (LMS)

EDUCATION (LMS)

Educational degrees can be complicated, but your software doesn't need to be.

SALES

SALES

Sell faster with simple but powerful Open Source Retail ERP Software

MANUFACTURING

MANUFACTURING

Move beyond legacy tools with a smart cloud manufacturing ERP software

INVENTORY

INVENTORY

Know where your business is

CRM

CRM

Close sales faster with the World's Best Open Source CRM

PROJECT MGMT.

PROJECT MGMT.

Upgrade your work style with an Open Source Project Management Software