使用Terraformer导出AWS上现有资源

发表于 评论数： 本文字数： 1.4k 阅读时长 ≈ 5 分钟

　　众所周知，通常情况下 Terraform 仅能编排通过它创建的资源， 那么怎样批量纳管现网非Terraform编排的资源呢？Terraformer 等工具就是来做这个事的。 Terraformer 是一个将现有的基础设施导出生成 Terraform file(.tf 以及 .tfstate) 的命令行工具，与 Terraform 刚好相反（Terraform: code -> infra， Terraformer: infra -> code）。

注意： Terraformer 目前依旧存在许多问题，比如状态同步不一致、terraform版本支持不及时等，因此如要生产可用还需要做更多的测试评估。为避免对现网生产环境造成不可预估的后果，请在独立的测试环境账号下测试评估！

环境说明

Terraform： v1.2.1
Terraformer： v0.8.20
aws CLI : v2.5.8

配置AWS profile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# aws configure sso

SSO start URL [None]: https://xxxxx.awsapps.com/start#/  # 这里输入SSO登录页地址
SSO Region [None]: ap-southeast-1                  # 配置sso默认Region
There are 8 AWS accounts available to you.         # 这里会让你选择要使用的sso账号，选定后回车即可  
Using the account ID xxxxxx
The only role available to you is: DveOps-Role    # 显示你拥有的role
Using the role name "DveOps-Role"                 # 使用该role
CLI default client Region [ap-southeast-1]:       # 配置默认Region
CLI default output format [yaml]:                 # 输出格式，默认即可
CLI profile name [DveOps-Role]: Alliot-DevOps     # 为profile创建别名 

To use this profile, specify the profile name using --profile, as shown:

aws s3 ls --profile Alliot-DevOps                 # 告诉你测试的方法

初始化Terraform插件

　　根据官方文档 需要初始化 AWS 插件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
mkdir aws_terraform
cd !$

# 编写provider配置
cat <<EOF >./main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }

  required_version = ">= 0.14.9"
}

provider "aws" {
  profile = "Alliot-DevOps"
  region  = "ap-southeast-1"
}
EOF

terraform init  # 初始化

　　初始化的过程中会去联网下载 aws Provider，成功后会返回类似如下的信息：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 3.27"...
- Installing hashicorp/aws v3.75.1...
- Installed hashicorp/aws v3.75.1 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

导出资源

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# terraformer import aws --resources=vpc,subnet --profile Alliot-DevOps

2022/05/06 17:24:36 aws importing default region
2022/05/06 17:24:40 aws importing... vpc
2022/05/06 17:24:41 aws done importing vpc
2022/05/06 17:24:41 aws importing... subnet
2022/05/06 17:24:41 aws done importing subnet
2022/05/06 17:24:41 Number of resources for service vpc: 1
2022/05/06 17:24:41 Number of resources for service subnet: 3
2022/05/06 17:24:41 Refreshing state... aws_subnet.tfer--subnet-022fd4fb959d54a64
2022/05/06 17:24:41 Refreshing state... aws_subnet.tfer--subnet-0508e858fa720baf6
2022/05/06 17:24:41 Refreshing state... aws_subnet.tfer--subnet-0306845c89930a562
2022/05/06 17:24:41 Refreshing state... aws_vpc.tfer--vpc-0bf213acc380519b2
2022/05/06 17:24:43 Filtered number of resources for service vpc: 1
2022/05/06 17:24:43 Filtered number of resources for service subnet: 3
2022/05/06 17:24:43 aws Connecting....
2022/05/06 17:24:43 aws save vpc
2022/05/06 17:24:43 aws save tfstate for vpc
2022/05/06 17:24:43 aws save subnet
2022/05/06 17:24:43 aws save tfstate for subnet

导出后的目录结构为：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
|-- generated
|   `-- aws
|       |-- subnet
|       |   |-- outputs.tf
|       |   |-- provider.tf
|       |   |-- subnet.tf
|       |   |-- terraform.tfstate
|       |   `-- variables.tf
|       `-- vpc
|           |-- outputs.tf
|           |-- provider.tf
|           |-- terraform.tfstate
|           `-- vpc.tf
`-- main.tf

　　我们进入 ./generated/aws/vpc 路径下预览一下：

1
2
3
cd generated/aws/vpc
# 需要先初始化 
terraform init

　　这个时候会出现如下报错：

1
2
3
4
5
6
7
8
9
10
# terraform init

Initializing the backend...
╷
│ Error: Invalid legacy provider address
│ 
│ This configuration or its associated state refers to the unqualified provider "aws".
│ 
│ You must complete the Terraform 0.13 upgrade process before upgrading to later versions.
╵

通过 “Invalid legacy provider address” error on Terraform - stackoverflow 中找到解决办法：

1
terraform state replace-provider -- -/aws hashicorp/aws

输入 “yes” 完成 provider 的替换后重新初始化：

1
terraform init

初始化完成后，再执行 terraform plan 进行配置预览，这时候又出现：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
terraform plan
provider.aws.region
  The region where AWS operations will take place. Examples
  are us-east-1, us-west-2, etc.

  Enter a value: ap-southeast-1

╷
│ Error: error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.
│ 
│ Please see https://registry.terraform.io/providers/hashicorp/aws
│ for more information about providing credentials.
│ 
│ Error: NoCredentialProviders: no valid providers in chain
│ caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
│ SharedCredsLoad: failed to load profile, .
│ EC2RoleRequestError: no EC2 instance role found
│ caused by: RequestError: send request failed
│ caused by: Get "http://x.x.x.x/latest/meta-data/iam/security-credentials/": dial tcp x.x.x.x:80: i/o timeout (Client.Timeout exceeded while awaiting headers)
│ 
│ 
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on <input-prompt> line 1:
│   (source code not available)
│ 
╵

　　这是鉴权的问题，未知原因，通过配置环境变量（即export AWS_PROFILE=Alliot-DevOps）指定 profile 并不可靠，有时候不生效。因此，在执行 terraformer 的时候可以通过参数 --profile=Alliot-DevOps 或修改 provider 代码块来指定 profile。 ，参考： How to fix “NoCredentialProviders: no valid providers in chain. Deprecated.”? - serverfault
类似上面提到的 “
初始化Terraform插件
”，编辑 provider.tf 中的 provider 块：

1
2
3
4
5
provider "aws" {
# 这里指定 profile 与 region
  profile = "Alliot-DevOps"
  region  = "ap-southeast-1"
}

重新 terraform plan 即可。

报错提示

未初始化(terraform init)

1
2
3
4
# terraformer import aws --resources=vpc,subnet  --regions=ap-southeast-1 --profile Alliot-DevOps

2022/05/06 16:33:42 aws importing region ap-southeast-1
2022/05/06 16:33:42 open /home/alliot/.terraform.d/plugins/linux_amd64: no such file or directory

profile 配置错误：

1
2
3
4
5
6
7
8
# terraformer import aws --resources=vpc,subnet  --regions=ap-southeast-1 --profile Alliot-DevOps

2022/05/06 17:24:11 aws importing default region
2022/05/06 17:24:17 aws importing... vpc
2022/05/06 17:24:22 aws error initializing resources in service vpc, err: no EC2 IMDS role found, operation error ec2imds: GetMetadata, request canceled, context deadline exceeded
2022/05/06 17:24:22 aws importing... subnet
2022/05/06 17:24:27 aws error initializing resources in service subnet, err: no EC2 IMDS role found, operation error ec2imds: GetMetadata, request canceled, context deadline exceeded
2022/05/06 17:24:27 aws Connecting....

20240331更新

从 Terraform v1.5.0 开始，我们可以直接使用 import block 来导入资源，同时生成资源代码块。
Import
如需要导入指定的 EC2 instance， 仅需要在 tf 文件中加入 import block：

1
2
3
4
import {
  to = aws_instance.example
  id = "i-abcd1234"
}

执行：

1
terraform plan -generate-config-out=generated.tf

将会自动生成资源代码片断到 generated.tf 文件中。