Aws Sourcetree

Blog Post: DevOps just got a whole lot easier with Opsgenie and AWS CloudFormation Registry and CLI. Opsgenie is a proud launch partner of the AWS CloudFormation Registry and CLI, an extension of CloudFormation. Users can now leverage Opsgenie’s support to better their IaC capabilities in their AWS. Sourcetree is a free Git desktop client for developers on Windows. Say goodbye to the command line and use the full capabilities of Git through Sourcetree’s beautifully simple interface. A simple, powerful Git client Sourcetree for Windows simplifies how you interact with. AWS Codecommit can be used as storage for private GIT repositories. The setup involves a few steps, assuming you have a valid AWS account already. Sign up for AWS Codecommit. Currently only region us-east-1 is available. Create a IAM user who will have access to the repositories, eg codecommit-user.

To add your supply request file, do the following:

  1. From your BitbucketStationSupplies in Bitbucket, click Source to open the source directory. Notice you only have one file, supplies.txt, in your directory.

    • A. Source page: Click the link to open this page.

    • B. Branch selection: Pick the branch you want to view.

    • C. More options button: Click to open a menu with more options, such as 'Add file'.

    • D. Source file area: View the directory of files in Bitbucket.

  2. From the Source page, click the More options button in the top right corner and select Add file from the menu. The More options button only appears after you have added at least one file to the repository. A page for creating the new file opens, as shown in the following image.

    • A. Branch with new file: Change if you want to add file to a different branch.

    • B. New file area: Add content for your new file here.

  3. Enter supplyrequest in the filename field.

  4. Select HTML from the Syntax mode list.

  5. Add the following HTML code to the text area:

    We are requesting additional supplies. Please send us the following:



    • space ice cream

    • nerf darts

    • telescope light shield

  6. Click Commit. The Commit message field appears with the message: supplyrequest created online with Bitbucket.

  7. Click Commit under the message field.

20 Feb 2019
💬 EN

Table of Contents

  • Summary
  • Running your first working git command against AWS

Do you have “single sign-on” for your Amazon Web Services accounts, but need to set up your computer to talk to their CodeCommit git repository hosting?
If you have Windows, Python, and Git installed on your computer, here’s how you can do so without messing up any other plain-old-username-and-password repositories you connect to!

I didn’t figure any of this out – I’m just transcribing it “for dummies”
(i.e. my future self when my computer gets replaced).

Summary

Here’s the guts of what we’re going to do to tell Git how to talk to Amazon:

  1. We’ll set up an easy way of putting fresh contents into c:usersMY_USERNAME.awscredentials once an hour
    (that is, we’ll make it easy to run a 300-line Python script dedicated to giving us a nice little “username” & “password” prompt)
  2. We’ll tell our Git installation that any repositories hosted at AWS CloudCommit should use the AWS CLI to facilitate login, not a username and password, but to go ahead and use username and password for everything else
  3. We’ll install the “AWS CLI” onto our computers and tell Git how to find it when executing the command “aws

Level: intermediate+

Not everything is covered here. I’m going to presume you have the experience necessary to do the following things on your own and that you have admin access to your computer to facilitate software installation:

  1. Use command lines comfortably
  2. Set up Git / SourceTree and establish a working relationship between a repository on your hard drive and an ordinary “remote” repository on another machine (when connecting to the “remote” is as simple as entering a username and password as prompted by Git or SourceTree, anyway)
  3. Edit your Windows “PATH” environment variables
  4. Set up Python on your computer and demonstrate that your computer is capable of executing print('Hello World')(even if just through an IDE like Spyder)
  5. Install Python modules you don’t yet have as part of your Python installation with the pip command in whatever command-line environment you installed / configured to be able to run pip(e.g. “Anaconda Prompt”)

Getting the URL for your repository

  1. There’s just “N. Virginia” (“East 1”) where all of CodeCommit lives now. So the URL to see your repositories is:
    https://console.aws.amazon.com/codecommit/home?region=us-east-1
  2. Although first, to log in, you may have to go to:
    https://SOME-SORT-OF-WEB-SITE-YOUR-COMPANY-GIVES-YOU.com
  3. In the left nav, expand “Source” and click on “Repositories”
    https://console.aws.amazon.com/codesuite/codecommit/repositories?region=us-east-1
  4. Find a repository you have access to and click it.
    https://console.aws.amazon.com/codesuite/codecommit/repositories/OUR_AWESOME_REPOSITORY/browse?region=us-east-1
  5. Far right, near the top of the page but not in the top nav, there’s a button with a drop-down that says “Clone URL.”
    Drop it down and pick “Connection Steps.”
    https://console.aws.amazon.com/codesuite/codecommit/repositories/OUR_AWESOME_REPOSITORY/connect?region=us-east-1
  6. If you see a warning at the top of the instructions with the following text, this blog post is for you!

    You are signed in using federated access or temporary credentials. The only supported connection method for these sign-in types is to use the credential manager included with the AWS CLI, as documented below. To configure a connection using SSH or Git credentials over HTTPS, sign in as an IAM user.

  7. There should still be a “Clone URL” drop-down in about the same area of the page as it was on the last page.
    Drop it down and pick “Clone HTTPS.”
  8. A green alert bar should appear at the top of the page under the top-nav saying:

    Copied

    https://git-codecommit.us-east-1.amazonaws.com/v1/repos/OUR_AWESOME_REPOSITORY

  9. Put that URL for your repository somewhere … you’ll need it later.

Install Git & SourceTree on your computer, if you haven’t

Test that your Git setup is working with some “ordinary” remote repository you’ve already had working in the past.

  • Use “git fetch --all” with the Git Bash command-line interface, make sure you get prompted for a password, and save
  • Edit a post in a readme or a blog, pushing the changes to SourceTree, and make sure they go live

Get all that sorted out if even that doesn’t work.

Edit your Git installation’s “global” GitConfig file

You’re going to need to edit your “global” GitConfig file to include these lines:

Control

Mine was stored in the C:usersMY_USERNAME folder, and was a file called .gitconfig

Test your git setup with “ordinary” remotes again to make sure you didn’t break anything.

(Close & re-open SourceTree after saving your GitConfig contents and before testing.)

Instead of “helper = ,” you can say “helper = wincred” for GitHub and BitBucket if you don’t want the Git Bash to prompt you for a password every time you do a fetch, push, pull, clone, etc.

If you ever realize you regret storing your password for a given account in the Windows Credential Manager, just type the Windows Key, start searching “manage windo…,” and click the Control Panel app “Manage Windows Credentials.” You can go through the list and delete your credentials against a given repository there.

Getting an hourly token from AWS

Tim Sullivan built a handy Python script that lets you fetch a fresh credential from AWS and dump it onto the part of your hard drive that AWS’s “command line interface” tool looks for such things, called “aws_saml_auth.py.”

(You’ll need to edit the value of idpentryurl on line 34, as the script on GitHub is tailored to the employer he was working for when he wrote the script.)

I don’t have “Python” added to my path, so running a given Python script from my command line looks a bit absurd right now:

If I put Python into my Windows “PATH” environment variable and executed the command from within the same folder that I’d put “aws_saml_auth.py,” I could have simply executed the following command:

Presuming my Python installed has all the modules that “aws_saml_auth.py” refers to installed (look up pip install), running this script on a command-line interface should ask me for my username & password (it’s my single-sign-on credentials for checking my work e-mail … with the “@CompanyName.com” at the end of the username).

If I type the correct values, I’ll see the following output, telling me that there’s a new folder on my hard drive under C:UsersMY_USERNAME called .aws with a file called credentials(no file extension) in it whose contents will be good for signing into Amazon for the next hour (after which, I’ll have to run this Python script again to generate fresh contents for “credentials“):

Note that it says the word “default” in lines 1 and 4 because that’s what “aws_saml_auth.py” says to do in line 87: ????

You’ll need to to run this Python script once an hour or so.

Therefore, you might find it handy to add a command to your command-line environment called “awslogin” as a shortcut to pull up the username & password prompt any time you notice “git fetch,” etc. isn’t working against an AWS-hosted repository that uses “federated” login.

My colleague set it up as a command called “login” that only worked within the “Git Bash” command-line interface by going to C:Program FilesGitusrbin and creating a new file in it called “login(no file extension) whose contents are:

(In his case, he had saved “aws_saml_auth.py” into C:Program FilesGitusrbin, and “python” was part of his PATH.)

Installing the AWS CLI so that Git’s attempt to execute the command “aws codecommit credential-helper” actually works

Now we need to install the AWS CLI so that Git’s attempt to execute the command “aws codecommit credential-helper” actually works.

Otherwise, Git Bash / SourceTree will dutifully follow our instructions to “use the command ‘aws’ instead of username and password for AWS CloudCommit remotes,” then “fall back” to prompting for a username & password (which won’t work, because Amazon doesn’t allow that for “federated” logins).

Let’s download the AWS CLI installer, install it, and add “aws” to our PATH environment variable.

In my case, the AWS CLI installer took care of the PATH environment variable for me: it added C:Program FilesAmazonAWSCLIbin to the system-wide PATH.

If I open up a standard Windows command prompt (Windows key -> type “cmd” and hit Enter), and if I type “aws --version,” I get an answer about what version of the AWS CLI is installed on my computer.

However, Git still doesn’t quite understand what “aws” is supposed to mean.

I can see that I have this problem because if I open up the “Git Bash” CLI and type “aws --version,” I got an “aws: command not found” complaint.

To fix that, I added a file called “aws(with no file extension) to C:Program FilesGitbin whose contents are:

(Thanks, Stack Overflow.)

I think that’s what made it suddenly start working, anyway.

I’m a little nervous that isn’t it, because now that I’ve gotten it working, if I erase the contents of this file, aws --version still works.

Anyway, as soon as “aws --version” works from the Git Bash CLI, I should be good to go!

  1. I have a way of putting fresh contents into c:usersMY_USERNAME.awscredentials once an hour
  2. I’ve told Git that any repositories hosted at AWS CloudCommit use the AWS CLI to facilitate login, not a username and password, but to go ahead and use username and password for everything else
  3. I’ve installed the AWS CLI onto my computer and told Git how to find it

Detecting and fixing expired credentials

Unfortunately, getting all this set up and blogging about it took so much troubleshooting to figure out, my 1-hour credentials from Amazon expired, so I may need to run “aws_saml_auth.py” from a command line again and update the contents of c:usersMY_USERNAME.awscredentials with a fresh token.

Expired credentials looked like this when doing a git fetch --all in the Git Bash CLI against a repo I’d gotten everything working for earlier:

In SourceTree, right-clicking on “origin” beneath “Remotes” in the left-nav and clicking “Fetch from origin” gave me the following error:

A successful fetch in the Git Bash looks like this:

Sourcetree Aws Codecommit Ssh

And in SourceTree, the progress bar just goes away, unless you have “Show Full Output” checked, in which case the output looks like this:

Running your first working git command against AWS

Of course, it was a bit presumptuous of me to presume you’d be doing a “git fetch” – that implies you had a remote-local pair set up in the first place. :-)

Remember that URL we got from Amazon waaaaay back at the beginning?

How to Run PHP File in Xampp or Wamp in Window Linux & Mac .... Let’s set up 2 local repositories on our computer, each using it as a “remote” – one with the “Git Bash” CLI (or with our Windows CLI if we set it up to be able to execute “git” commands when we installed Git), and the other with SourceTree.

Clone with Git Bash

  1. Create a new folder on your hard drive, c:example
  2. Do cd c:example in the Git Bash command prompt
  3. Do git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/OUR_AWESOME_REPOSITORY aws_test_1

Aws Source Code Management

The output should look something like this, and you should have a new c:exampleaws_test_1 folder on your computer. There should be files in it, if there were files in the remote.

Clone with SourceTree

  1. Click the “+” by the tabs of your open repositories, near the top, and in the big buttons near the top of the “Local Repositories” screen, click the “Clone” button with an icon that looks like an arrow pointing into an open box.
  2. In “Source Path / URL,” type: https://git-codecommit.us-east-1.amazonaws.com/v1/repos/OUR_AWESOME_REPOSITORY
  3. In “Destination Path,” type: C:exampleaws_test_2
  4. In “Name,” type: Best AWS Test 2
  5. Click the “Clone” button

If you don’t check the “Show Full Output” checkbox of the progress bar, it might just disappear when done. But if you do, its log will look something like this:

In the left nav, if you click “Branches” and click “master” (or whatever your branch is called), you should be able to see the full commit-log history from the remote copy of the repository (that is, AWS), presuming there is any history.

Aws Source Destination Checks

You should have a new c:exampleaws_test_2 folder on your computer. There should be files in it, if there were files in the remote.

CONGRATULATIONS, YOU’RE IN!

Happy git-ting!

Further Resources

  • Using CodeCommit and GitHub Credential Helpers by James Wing