terraform-oci-arch-lakehouse-mds

Reference Architecture

This Terraform code spins up one or more Oracle Cloud Infrastructure (OCI) instances, a shared file system and a load balancer when 2 or more instances are used, creates the required OCI networking components, deploys Zeppelin and Grafana on the instance(s), deploys MySQL HeatWave, an Object Storage bucket and a Data Integration workspace.

Architecture Diagram

Prerequisites

Permission to manage the following types of resources in your Oracle Cloud Infrastructure tenancy: vcns, internet-gateways, route-tables, security-lists, subnets, mysql-family, buckets and instances.

Quota to create the following resources: 1 VCN, 2 subnets, 1 Internet Gateway, 1 NAT Gateway, 2 route rules, 1 MySQL Database System (MDS) instance, 1 Object Storage bucket and 1 (or more) compute instance(s) for Analytics.

If you don't have the required permissions and quota, contact your tenancy administrator. See Policy Reference, Service Limits, Compartment Quotas.

Deploy Using Oracle Resource Manager

Click



If you aren't already signed in, when prompted, enter the tenancy and user credentials.

Review and accept the terms and conditions.

Select the region where you want to deploy the stack.

Follow the on-screen prompts and instructions to create the stack.

After creating the stack, click Terraform Actions, and select Plan.

Wait for the job to be completed, and review the plan.

To make any changes, return to the Stack Details page, click Edit Stack, and make the required changes. Then, run the Plan action again.

If no further changes are necessary, return to the Stack Details page, click Terraform Actions, and select Apply.

Deploy Using the Terraform CLI

Clone the Module

Now, you'll want a local copy of this repo. You can make that with the commands:

    git clone https://github.com/oracle-devrel/terraform-oci-arch-lakehouse-mds.git
    cd terraform-oci-arch-lakehouse-mds
    ls

Prerequisites

First off, you'll need to do some pre-deploy setup. That's all detailed here.

Create a terraform.tfvars file, and specify the following variables:

# Authentication
tenancy_ocid        = "<tenancy_ocid>"
user_ocid           = "<user_ocid>"
fingerprint         = "<finger_print>"
private_key_path    = "<pem_private_key_path>"

region              = "<oci_region>"
compartment_ocid    = "<compartment_ocid>"

# MySQL and Compute variables
admin_password      = "<MySQL_admin_password>"
db_password         = "<db_user_password>"
numberOfNodes       = 1 # value 2+ for multinode scenario will be deployed inluding LB & FSS.

Create the Resources

Run the following commands:

terraform init
terraform plan
terraform apply

Destroy the Deployment

When you no longer need the deployment, you can run this command to destroy the resources:

terraform destroy

Testing your Deployment

After the deployment is finished, you can access Zeppelin and Grafana from the URLs found in the Terraform output.

zeppelin_home_URL = http://203.0.113.1/
grafana_home_URL = http://203.0.113.1:3000/

Contributing

This project is open source. Please submit your contributions by forking this repository and submitting a pull request! Oracle appreciates any contributions that are made by the open source community.

Attribution & Credits

This repository was initially inspired on the materials found in lefred's blog. For that reason we would sincerely like to thank Frédéric Descamps (https://github.com/lefred). We would also like to thank Luke Feldman (https://github.com/lfeldman) for all his help and dedication.

License

Copyright (c) 2022 Oracle and/or its affiliates.

Licensed under the Universal Permissive License (UPL), Version 1.0.

See LICENSE for more details.