Jekyll2022-10-19T10:26:18+00:00http://codingo.io/feed.xmlCodingoInstructional content aimed at those passionite about bug bounties, and penetration testing
Contributing to Github for Information Security Professionals2020-10-08T07:01:58+00:002020-10-08T07:01:58+00:00http://codingo.io/tools/github/bounty/2020/10/08/github-security-professionals<p>Every Month if <a href="https://hacktoberfest.digitalocean.com/">Hacktoberfest</a>, which, if you’re not already familiar with is an event run by Digital Ocean to reward open source contributions with swag (normally a t-shirt and stickers). All in all, it’s a very well received event that provides the perfect time to jump into open source, if you haven’t already.</p>
<p>The intention of this guide is to arm you with the knowledge of how you can get involved, even if you’re not entirely familiar with programming, or if you are, and you want a reference point for “how to github” - I’m aiming for this to land with both.</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/AQEOz5zbW4w" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
<h1 id="table-of-contents">Table of Contents</h1>
<ul>
<li><a href="#what-is-github-">What is Github?</a>
<ul>
<li><a href="#but-what-is-git-">But what is Git?</a></li>
</ul>
</li>
<li><a href="#how-do-i-get-started-">How do I get started?</a></li>
<li><a href="#visiting-a-project--repository-">Visiting a Project (repository)</a></li>
<li><a href="#creating-your-first-project">Creating your first project</a></li>
<li><a href="#master-and-main-branches">Master and Main Branches</a></li>
<li><a href="#commiting-direct-to-main-branch">Commiting direct to Main Branch</a></li>
<li><a href="#authentication">Authentication</a></li>
<li><a href="#projects-for-non-coders">Projects for non-coders</a>
<ul>
<li><a href="#discloseio">Disclose.io</a></li>
<li><a href="#can-i-take-over-xyz">Can-I-Take-Over-XYZ</a></li>
<li><a href="#keyhacks">Keyhacks</a></li>
<li><a href="#infosec-community-repositories">Infosec Community Repositories</a></li>
</ul>
</li>
</ul>
<h1 id="what-is-github">What is Github?</h1>
<p>Glad you asked! <a href="https://github.com/">Github</a> is self purported to be the worlds leading software development platform. In my own words, it’s a programming social network where you can receive the benefit of open source projects others have contributed to, contribute to yourself, or host your own code upon. This isn’t just limited to code also, some informational resources can be found here, completely community driven.</p>
<p>Without question, Github is the biggest of these platforms and although <a href="https://about.gitlab.com/">Gitlab</a> and others exist, the security community has never really gravitated towards them, with the very, very vast majority of projects exisitng on and being powered by Github.</p>
<h2 id="but-what-is-git">But what is Git?</h2>
<p>Both Github and Gitlab are websites that wrap around git, a version control management system. This is well summarised below however don’t worry if some of this seems daunting, as I’m aiming to cover the “how” throughout this guide, leaving you armed with enough information to jump in and get involved.</p>
<iframe width="1203" height="677" src="https://www.youtube.com/embed/2ReR1YJrNOM" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
<h1 id="how-do-i-get-started">How do I get started?</h1>
<p>After registering an account and poking around, it’s worth understanding a few “how to” elements of Github. Let’s use https://github.com/codingo for examples sake.</p>
<p>When you first visit this, you’ll see the following:</p>
<p><img src="http://codingo.io/assets/github-codingo.png" alt="github-codingo.png" /></p>
<p>The “pinned” section is a summary of repositories that I’ve chosen to highlight. A respository is essentially the term used to house git repositories, or version control sytems. If a git repository contains a “README.md” then that will be displayed when you first open the repository, as shown below:</p>
<p><img src="http://codingo.io/assets/interlace-readme.png" alt="interlace-readme.png" /></p>
<p>As you can also see, the files within the repository will be shown, along with the latest commit messages against those files (we’ll dig into those later), and the date they were last modified. You can also click these, to see what those changes were, and who they were made by. In essence, a repository stores a full history of these files, creating an immutable (to a degree, you can force change if you’re the project owner) record of that history, should you need to ever track down the origin of a change, or revert it.</p>
<p>Back to profiles, you’ll also see different elements such as badges, organisations that user belongs to and more. The “highlights” a user can have come from different activities over Github itself. For example, the “Security Bug Bounty Hunter” highlight on my profile is one I’m particularly proud of, as it shows that I’ve found and reported a security issue in Github itself. Likewise, the organisation I work for (Bugcrowd) contributes to the open source space and I’ve been invited to that organisation on Github, my affiliation is shown on my profile. All of this to say, you can build your profile out with other actions over Github, beyond just contributing to open source projects.</p>
<h1 id="visiting-a-project-repository">Visiting a Project (repository)</h1>
<p>Moving beyond profiles, you should dig into code repositories. In github, these are fully hosted pages, showing the key information around a repository. If we go to https://github.com/codingo/Interlace, we can break this down further.</p>
<p>This repository contains a number of elemnts. Front an centre you’ll see the files and folders relating to the project, as well as when they were last updated.</p>
<p><img src="http://codingo.io/assets/github-project-page.png" alt="github-project-page.png" /></p>
<p>Clicking the descriptions next to these files will show you the details of the “commit” that description is referring to. Essentially, a commit is a change to a file (sent via a pull request). We’ll dig into this more later, but what you need to know about git, and by extension then Github is that every change to a file, folder, or otherwise is stored in a commit and the entire repository is a database of sorts storing the commit history for a project.</p>
<p>The power of this commit history is that you can see the full history of a file (this is referred to as “git blame”, but it doesn’t have a negative conotiation as the wording suggests). This means that if you make a mistake, or you want to note when a change occured, you can do it. The other power of git, and its intention, is that this history, stored with the way git merges changes to projects (more on this later also) allows for small and large teams alike to work on the same project in paralell, without getting in each others way (at least, not intentionally!).</p>
<h1 id="creating-your-first-project">Creating your first project</h1>
<p>I’m going to assume you’ve registered a github account, logged in, and want to start with creating your first project. So let’s! Firstly, to do that login, and at the profile page, click “Repositories” (found on the top bar).</p>
<p><img src="http://codingo.io/assets/github-repositories.png" alt="github-repositories.png" /></p>
<p>At the next page, click “New” and give your repository a name. These will also drive the URL to your project, and it’s best to be succinct. You’ll also need to provide a description, and assuming you don’t have a premium (or student) account on Github, you will need to make a public repository.</p>
<p>I also suggest initializing the repository with content, the best way I find to do this is to select “Add a README file”. We’ll dig into editing and updating using this file as an example later, so it’s best to select this to more easily follow this guide.</p>
<p><img src="http://codingo.io/assetsnew-repository.png" alt="new-repository.png" /></p>
<p>After you’ve done that, click “create repository”, and we’re off!</p>
<h1 id="master-and-main-branches">Master and Main Branches</h1>
<p>As you create pull requests you’re performing work on your own branch, which you then propose to the owners of the repository in a pull request. Essentially, you’re saying, “Hey, I did this work over here, but the main project should add it as it is good for others”.</p>
<p>The primary branch that a project works on historically has been referred to as the master branch, however this term has now shifted to referring to this as the main branch, and Github is slowly making steps towards applying this site-wide.</p>
<h1 id="commiting-direct-to-main-branch">Commiting direct to Main Branch</h1>
<p>You can, as the owner of a repository also commit changes directly to the main branch, instead of going through a pull request. This is only recommended for every minor changes, if at all, as every time you do so on a Github project you’re also risking a user cloning the repository at a point in time where you have functionally broken or untested code. For that reason, any user who isn’t a listed contributor (a permission setting, not literally a contributor) to a repository must go through a pull request to submit changes.</p>
<h1 id="authentication">Authentication</h1>
<p>Authentication is key if you’re working with Github, and can be found discussed at length here: https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github</p>
<h1 id="projects-for-non-coders">Projects for non-coders</h1>
<h2 id="discloseio">Disclose.io</h2>
<p>The Disclose.io project tracks websites operating disclosure programs, and the terms under whcih they operate. It’s community driven, and can be found at: https://github.com/disclose</p>
<h2 id="can-i-take-over-xyz">Can-I-Take-Over-XYZ</h2>
<p>The can-i-take-over-xyz list tracks services which are, or aren’t vulnerable to subdomain takeover in a discussion type format. You can find it at https://github.com/edoverflow/can-i-take-over-xyz</p>
<h2 id="keyhacks">Keyhacks</h2>
<p>The Keyhacks repository tracks safe methods for checking if an API key discovered in a pentester, or Bug Bounty program is valid or not. You can find it at https://github.com/streaak/keyhacks/</p>
<h2 id="infosec-community-repositories">Infosec Community Repositories</h2>
<p>The Infosec Community Github organisation tracks a number of informational repositories you can contribute to, found here: https://github.com/infosec-community</p>Every Month if Hacktoberfest, which, if you’re not already familiar with is an event run by Digital Ocean to reward open source contributions with swag (normally a t-shirt and stickers). All in all, it’s a very well received event that provides the perfect time to jump into open source, if you haven’t already.Everything you need to know about FFUF2020-09-17T07:01:58+00:002020-09-17T07:01:58+00:00http://codingo.io/tools/ffuf/bounty/2020/09/17/everything-you-need-to-know-about-ffuf<h1 id="summary">Summary</h1>
<p>This guide is a large summary of the information security tool, FFUF. This is also paired with a video companion guide, shown below:</p>
<iframe width="978" height="550" src="https://www.youtube.com/embed/iLFkxAmwXF0" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
<h1 id="table-of-contents">Table of Contents</h1>
<ul>
<li><a href="#other-sources---credit">Other Sources / Credit</a></li>
<li><a href="#before-we-start">Before we start</a></li>
<li><a href="#what-is-ffuf--and-what-is-it-used-for-">What is FFUF, and What is it used for?</a>
<ul>
<li><a href="#who">Who</a></li>
<li><a href="#what">What</a></li>
<li><a href="#where">Where</a></li>
<li><a href="#why">Why</a></li>
<li><a href="#command-line-driven-applications">Command Line Driven Applications</a></li>
</ul>
</li>
<li><a href="#installation">Installation</a>
<ul>
<li><a href="#install-from-source">Install from Source</a></li>
<li><a href="#upgrading-from-source">Upgrading from Source</a></li>
<li><a href="#kali-linux-apt-repositories">Kali Linux APT Repositories</a></li>
<li><a href="#other-locations---debian-unstable---snap--etc-">Other Locations - Debian Unstable / SNAP, etc’</a></li>
</ul>
</li>
<li><a href="#basic-usage">Basic Usage</a>
<ul>
<li><a href="#what-is-directory-brute-forcing-">What is Directory Brute Forcing?</a></li>
<li><a href="#what-is-a-wordlist-">What is a Wordlist?</a>
<ul>
<li><a href="#what-is-seclists-">What is SecLists?</a></li>
<li><a href="#what-wordlists-should-i-start-with-">What Wordlists should I start with?</a></li>
</ul>
</li>
<li><a href="#your-first-directory-brute-force">Your first Directory Brute Force</a></li>
</ul>
</li>
<li><a href="#recursion">Recursion</a></li>
<li><a href="#extensions">Extensions</a></li>
<li><a href="#fuzzing-multiple-locations">Fuzzing Multiple Locations</a>
<ul>
<li><a href="#wordlist-parameter-bug">Wordlist Parameter Bug</a></li>
</ul>
</li>
<li><a href="#handling-authentication">Handling Authentication</a>
<ul>
<li><a href="#cookie-based-authentication">Cookie Based Authentication</a></li>
<li><a href="#header-based-authentication">Header Based Authentication</a></li>
<li><a href="#more-complex-authentication-flows">More Complex Authentication Flows</a></li>
</ul>
</li>
<li><a href="#threads">Threads</a></li>
<li><a href="#using-silent-mode-for-passing-results">Using Silent Mode for Passing Results</a></li>
<li><a href="#error-handling-and-scan-tweaks">Error Handling and Scan Tweaks</a>
<ul>
<li><a href="#automatically-calibrate-filtering">Automatically Calibrate Filtering</a></li>
<li><a href="#custom-automatic-calibration-filtering">Custom Automatic Calibration Filtering</a></li>
<li><a href="#immediately-stop-on-error-case">Immediately Stop on Error Case</a></li>
<li><a href="#stop-on-spurious-errors">Stop on Spurious Errors</a></li>
</ul>
</li>
<li><a href="#request-throttling-and-delays">Request Throttling and Delays</a>
<ul>
<li><a href="#delay-between-requests">Delay Between Requests</a></li>
<li><a href="#limited-max-requests-second">Limited Max Requests/second</a></li>
</ul>
</li>
<li><a href="#match-options">Match Options</a>
<ul>
<li><a href="#match-on-response-code">Match on Response Code</a></li>
<li><a href="#match-on-regular-expression">Match on Regular Expression</a></li>
</ul>
</li>
<li><a href="#filter-and-matches">Filter and Matches</a></li>
<li><a href="#sending-ffuf-scans-via-burp-suite">Sending FFUF scans via Burp Suite</a>
<ul>
<li><a href="#locally--using-replay-proxy">Locally, Using Replay Proxy</a></li>
<li><a href="#using-an-interface">Using an Interface</a></li>
<li><a href="#remote-vps-traffic-via-a-reply-proxy">Remote VPS Traffic via a Reply Proxy</a></li>
</ul>
</li>
<li><a href="#advanced-wordlist-usage">Advanced Wordlist Usage</a>
<ul>
<li><a href="#clusterbomb">Clusterbomb</a></li>
<li><a href="#pitchfork">Pitchfork</a></li>
</ul>
</li>
<li><a href="#handling-output">Handling Output</a>
<ul>
<li><a href="#standard-output">Standard Output</a>
<ul>
<li><a href="#insiderphd-tool">InsiderPHD Tool</a></li>
</ul>
</li>
<li><a href="#html-output">HTML Output</a></li>
<li><a href="#using-silent-and-tee">Using Silent and Tee</a></li>
</ul>
</li>
<li><a href="#importing-requests">Importing Requests</a></li>
<li><a href="#contributing-to-this-guide">Contributing to this guide</a>
<ul>
<li><a href="#contributors">Contributors</a></li>
</ul>
</li>
</ul>
<h1 id="other-sources--credit">Other Sources / Credit</h1>
<p>Understandably, putting this guide and the associated video content together has taken quite a long time (in the order of months, as it’s my first steps into video). Throughout that time some other great creators have put out other content, I heavily recommend watching. This content has inspired this project further, and I don’t think it would be what it is without their input. Notably, and a video I recommend watching in addition to my own for a more complete picture is Katie Paxton-Fear’s <a href="https://www.youtube.com/watch?v=aN3Nayvd7FU">How to Use FFUF</a> YouTube video.</p>
<p><a href="https://www.youtube.com/watch?v=aN3Nayvd7FU"><img src="http://codingo.io/assets/2020-09-17-ffuf/InsiderPHD-how-to-use-ffuf.png" alt="InsiderPHD-how-to-use-ffuf.png" /></a></p>
<p>Also, a shoutout to <a href="https://mobile.twitter.com/Jhaddix">Jason Haddix</a>, <a href="https://mobile.twitter.com/stokfredrik">STÖK</a>, <a href="https://mobile.twitter.com/hakluke">hakluke</a>, <a href="https://mobile.twitter.com/InsiderPhD">InsiderPHD</a>, and <a href="https://mobile.twitter.com/joohoi">Joohoi</a> for helping answer my numerous questions and being a soundboard as I pulled this together.</p>
<h1 id="before-we-start">Before we start</h1>
<p>This guide is a reference point for using a web application security tool, FFUF. If you have a passion for this space, but the guide seems daunting, that doesn’t mean you can’t do this, it just means there’s some prerequisites to dive into first.</p>
<p>A great starting point is:</p>
<p><a href="https://www.youtube.com/watch?v=lZAoFs75_cs">Health Adams - Linux for Ethical Hackers</a> which will help you to understand the fundamentals of what’s happening here in the terminal, which should allow you to then make use of this guide.</p>
<p>In addition I also recommend Heath’s other content, available <a href="https://www.youtube.com/channel/UC0ArlFuFYMpEewyRBzdLHiw">on their YouTube channel</a> and I also recommend <a href="https://www.youtube.com/user/RapidBug">InsiderPHD’s</a> content as a very good starting point.</p>
<p>Other notable creators well worth watching on your hacking journey include (but certainly aren’t limited to):</p>
<ul>
<li><a href="https://www.youtube.com/channel/UCk0f0svao7AKeK3RfiWxXEA">Jason Haddix</a> and his streams for more tooling insight.</li>
<li><a href="https://www.youtube.com/channel/UClcE-kVhqyiHCcjYwcpfj9w">LiveOverflow</a> for amazing insights into just how deep this rabbit hole can go</li>
<li><a href="https://www.youtube.com/channel/UCq9IyPMXiwD8yBFHkxmN8zg">Farah Hawa</a> for a variety of hacking guides aimed at beginners</li>
<li><a href="https://www.youtube.com/channel/UCjBhClJ59W4hfUly51i11hg">The XSS Rat</a> for a range of content from CTF’s to thought process</li>
<li><a href="https://www.youtube.com/channel/UCQN2DsjnYH60SFBIA6IkNwg">STÖK</a> for the best hacking community orientated content around, and by far the coolest / well centred creator to learn from.</li>
<li><a href="https://www.youtube.com/user/TomNomNomDotCom/videos">TomNomNom</a> because learning to quit vim is vital, and you may learn some other tricks along the way</li>
<li><a href="https://www.youtube.com/channel/UCCzvz8jsulXm27Cd6k3vzyg">Hakluke</a> for direction on mindset, how to approach bug bounties, and industry insights.</li>
</ul>
<p>And many more, that I’m sure to have missed, but not intentionally. I love you all.</p>
<h1 id="what-is-ffuf-and-what-is-it-used-for">What is FFUF, and What is it used for?</h1>
<h2 id="who">Who</h2>
<p>Being an open source project, FFUF is maintained by the community however notably, it’s founder and principal maintainer, joohoi puts countless hours into driving the project forward. If you FFUF useful, you can support the work here: https://github.com/sponsors/joohoi</p>
<h2 id="what">What</h2>
<p>Firstly, the “what” is quite important. FFUF, or “Fuzz Faster you Fool” is an open source web fuzzing tool, intended for discovering elements and content within web applications, or web servers. What do we mean by this? Often when you visit a website you will be presented with the content that the owner of the website wants to serve you with, this could be hosted at a page such as <code class="language-plaintext highlighter-rouge">index.php</code>. Within security, often the challenges in a website that need to be corrected exist outside of that. For example, the owner of the website may have content hosted at <code class="language-plaintext highlighter-rouge">admin.php</code>, that you both want to know about, and test. FFUF is a tool for uncovering those items, for your purusal.</p>
<h2 id="where">Where</h2>
<p>FFUF is maintained as public open source, and can be found at: https://github.com/ffuf/ffuf</p>
<p>This means anybody who wishes to contribute to FFUF, can, provided the maintainer (joohoi) accepts and “merges” the contributed changes back to the main project.</p>
<h2 id="why">Why</h2>
<p>Particularly in the bug bounty scene, a lot of people have gravitated towards FFUF since its release. Whilst a good majority of this shift is likely down to the following of the crowd, there is a definite portion of the community who have made the change due to FFUF’s speed, flexibility, and ability to quickly integrate into outside tooling. In addition, the maintenance of the project is top notch, especially when compared to compeitive offerings which although similar in features, lack the same passion and speed to market of new features that FFUF has consistenty shown. All of these things considered, FFUF is a mainstay in any toolkit, and this guide aims to emphasize why that should be, and will likely continue to be the case.</p>
<h2 id="command-line-driven-applications">Command Line Driven Applications</h2>
<p>FFUF is a command line driven application that runs in the Linux Terminal, or the Windows Command Prompt, meaning that it doesn’t contain an interactive GUI, and is instead powered by inputted command line flags. Whilst this may seem more limiting at first, this lends itself to a higher degree of flexibility as you can make full usage of the tool over remote servers, as well as “pipe” (pass to / from) into and out of FFUF with other command line driven tools. This may feel limiting at first, however I encourage you to follow through this guide to make the most of FFUF, and as you become a more advanced user, you will be able to make the most of this flexibility.</p>
<p>Often, you’ll hear FFUF compared to tools such as <a href="https://tools.kali.org/web-applications/dirb">dirb</a>, or <a href="https://tools.kali.org/web-applications/dirbuster">dirbuster</a>, which whilst true at a certain level, isn’t a very fair comparison. Whilst FFUF can be used to perform directory brute forcing it’s true power lies in its flexibility, and a better comparison tool for FFUF should be made against something like Burp Suite Intruder, or Turbo Intruder. We’ll aim to cover that flexiblity further throughout this guide.</p>
<h1 id="installation">Installation</h1>
<h2 id="install-from-source">Install from Source</h2>
<p>If you wish to install the latest stable build from the <code class="language-plaintext highlighter-rouge">main</code> branch of the ffuf project, you can do so with:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>go get github.com/ffuf/ffuf
</code></pre></div></div>
<p>After installing, ffuf will be available in <code class="language-plaintext highlighter-rouge">~/go/bin/ffuf</code>.</p>
<h2 id="upgrading-from-source">Upgrading from Source</h2>
<p>Much like compiling from source, upgrading from source is not much more complicated, with the only change being the addition of the <code class="language-plaintext highlighter-rouge">-u</code> flag. Upgrading from source should be done with:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>go get -u github.com/ffuf/ffuf
</code></pre></div></div>
<h2 id="kali-linux-apt-repositories">Kali Linux APT Repositories</h2>
<p>If you’re using <a href="https://www.kali.org/">Kali Linux</a> you’ll find FFUF in the <a href="https://en.wikipedia.org/wiki/APT_(software)">apt</a> repositories, allowing you to install by running <code class="language-plaintext highlighter-rouge">sudo apt-get install ffuf</code>, this will present an output similar to the following:</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/ffuf-apt-install.png" alt="FFUF" /></p>
<p>After installation, you can verify the version installed by using:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -V
</code></pre></div></div>
<p>If you also installed from source you’ll note that the version you’re operating is not the same as the version in your <code class="language-plaintext highlighter-rouge">$GOPATH</code> (<code class="language-plaintext highlighter-rouge">~/go/bin</code>). APT builds are normally older, but considered more stable builds of applications however can be less feature rich because of this.</p>
<h2 id="other-locations---debian-unstable--snap-etc">Other Locations - Debian Unstable / SNAP, etc’</h2>
<p>As it becomes more widely used, more ways to install FFUF are becoming available. FFUF currently flows into Debian Unstable, in addition to some flavours of Ubuntu who use those source. It’s also available in the Fedora official repositories, and a SNAP integration is currently underway as well.</p>
<h1 id="basic-usage">Basic Usage</h1>
<h2 id="what-is-directory-brute-forcing">What is Directory Brute Forcing?</h2>
<p>At its core, one of the main functions that people use FFUF for, is directory brute forcing. With that in mind, let’s fuzz! Without passing custom values (covered later in this course), FFUF will replace the value of <code class="language-plaintext highlighter-rouge">FUZZ</code> with the value of your wordlist.</p>
<h2 id="what-is-a-wordlist">What is a Wordlist?</h2>
<p>What’s a wordlist? A wordlist is essentially a list of items in a text file, seperated by lines, that are tailor built around a purpose.</p>
<p>One of the best collections of wordlists, is <a href="https://github.com/danielmiessler/SecLists/">SecLists</a>. Curated by g0tm1lk, jhaddix and Daniel Miessler this collection has a wordlist for every occasion.</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/seclists-highlight.png" alt="FFUF" /></p>
<h3 id="what-is-seclists">What is SecLists?</h3>
<p>SecLists is managed on Github, so anyone can contribute to these lists and with such an active and well known repository, this leads to a flurry of beneficial contributions. To date, over 100 people have contributed to SecLists, with no sign of it slowing. As you further build in your Security knowledge, if you’ve made the most of SecLists I recommend aiming to give back, through a contribution, or by supporting Daniel, the project owner, through Github Sponsers, here: https://github.com/sponsors/danielmiessler.</p>
<h3 id="what-wordlists-should-i-start-with">What Wordlists should I start with?</h3>
<p>If you’re getting started in security, and you’re unsure where to start with wordlists, a good/safe collection of lists are the discovery wordlists in SecLists, specifically:</p>
<ul>
<li>https://github.com/danielmiessler/SecLists/blob/master/Discovery/Web-Content/directory-list-2.3-small.txt</li>
<li>https://github.com/danielmiessler/SecLists/blob/master/Discovery/Web-Content/directory-list-2.3-medium.txt</li>
<li>https://github.com/danielmiessler/SecLists/blob/master/Discovery/Web-Content/directory-list-2.3-big.txt</li>
</ul>
<p>As you progress in your journey, be sure to revisit this and look into tooling and workflow changes that allow you to both use more tailored wordlists for the asset that you’re approaching, as well as build custom wordlists around a target as you wish to get more information from it.</p>
<h2 id="your-first-directory-brute-force">Your first Directory Brute Force</h2>
<p>For this example, let’s create a simple wordlist. In this case, we’ll put the following items into it:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>test
test1
admin
panel
</code></pre></div></div>
<p>Save this in the same location where you intend to run FFUF from, as <code class="language-plaintext highlighter-rouge">wordlist.txt</code>.</p>
<p>For this example, we’ll also brute force against this website, codingo.io. FFUF takes two basic arguments that we need to use here, the first, <code class="language-plaintext highlighter-rouge">-u</code> is the target URL (in this case, codingo.io). The second, is <code class="language-plaintext highlighter-rouge">-w</code>, which is the path to the wordlist file(s) that we wish to make use of. You can specify multiple wordlists in a comma delimited list, if you so require. We also need to put the word <code class="language-plaintext highlighter-rouge">FUZZ</code> where we want our wordlist items to be placed. In this case, we’re aiming to brute force for new directories, so we put this after the URL.</p>
<p>Putting this altogether, the command for our first directory brute force will be:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://codingo.io/FUZZ -w ./wordlist.txt
</code></pre></div></div>
<p>When we run this, we should receive similar to the following:</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/ffuf-codingo-test.png" alt="FFUF" /></p>
<p>Note that from our three results, one has come back with a result. In this case, it’s come back with a <code class="language-plaintext highlighter-rouge">301</code> response code, which indicates a redirect is present here. As none of the other endpoints responded with this, instead opting for a <code class="language-plaintext highlighter-rouge">404</code> page not found response (which we don’t have set to match and display), we should investigate this. Doing so, shows the following:</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/ffuf-codingo-admin.png" alt="FFUF" /></p>
<p>Congratulations! You’ve just brute forced a website and discovered your first endpoint that isn’t present from the main page itself.</p>
<h1 id="recursion">Recursion</h1>
<p>Recursion is essentially performing the same task again, but in this context, at another layer. For example, in our item above, we identified an admin panel, but what if we want to scan further under that? One method, could be to scan again, but by changing our URL and fuzzing endpoint to the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://codingo.io/admin/FUZZ -w ./wordlist.txt
</code></pre></div></div>
<p>Now whilst this will acheive our goal, it doesn’t scale well. When bug hunting, we may find 20, 30, or even 100 directories, all which we want to explore at another level.</p>
<p>Enter, recursion. By setting the flag <code class="language-plaintext highlighter-rouge">recursion</code> we tell FFUF to take our scan, and apply another layer to it. A second flag, <code class="language-plaintext highlighter-rouge">recursion-depth</code> tells FFUF how many times to perform this action (for example, if we find another layer under admin, shoud we proceed to another layer or stop?). There are some caveats, however. In FFUF you can’t use customer fuzzing keywords with recursion, and you’re limited to the use of <code class="language-plaintext highlighter-rouge">FFUF</code>. Whilst this won’t matter for the vast array of applications it will limit usage when using pitchfork scanning modes, which we’ll cover later. This isn’t a significant issue, however, and just something to take a mental note of for future reference.</p>
<p>When we run this command again, but with the recursion flag, we can see the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://codingo.io/FUZZ -w ./wordlist -recursion
</code></pre></div></div>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/ffuf-recursion.png" alt="FFUF" /></p>
<p>In this case, both items “admin” and a subpage under that “panel” were discovered.</p>
<h1 id="extensions">Extensions</h1>
<p>Often when you find a directorty you’re also going to want to look for file extensions of that. This can be invaluable for finding bugs when there’s a zip file, or backup file of the same name.</p>
<p>Extensions in FFUF are specified with the <code class="language-plaintext highlighter-rouge">e</code> parameter and are essentially suffixs to your wordlist (as not all extensions start with a <code class="language-plaintext highlighter-rouge">.</code>). For example, expanding upon our original scan with the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://codingo.io/FUZZ -w ./wordlist -recursion -e .bak
</code></pre></div></div>
<p>This now presents new hits! As shown below:</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/ffuf-extension-hits.png" alt="FFUF" /></p>
<h1 id="fuzzing-multiple-locations">Fuzzing Multiple Locations</h1>
<p>By default, FFUF will only look for a single location to fuzz, donate by the term <code class="language-plaintext highlighter-rouge">FUZZ</code>. Reviewing our original example, this was the approach taken to FUZZ the directory name:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://codingo.io/FUZZ -w ./wordlist.txt
</code></pre></div></div>
<p>But what if we want to fuzz multiple locations? This can be acomplished by comining the ability to define what a fuzz location would be with a wordlist, as well as using multiple wordlists.</p>
<p>For example, in the following we’re using the term <code class="language-plaintext highlighter-rouge">W1</code> to fuzz our location, instead of <code class="language-plaintext highlighter-rouge">FUZZ</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://codingo.io/W1 -w ./wordlist.txt:W1
</code></pre></div></div>
<p>This runs the same scan as our previous example, except <code class="language-plaintext highlighter-rouge">W1</code> is now our insert instead of <code class="language-plaintext highlighter-rouge">FUZZ</code>. Now, let’s assume that instead of <code class="language-plaintext highlighter-rouge">codingo.io</code> we had identified multiple websites we wanted to check over at the same time. For that, we could create a wordlist of all of the domains we wanted to test, and use the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://W2/W1 -w ./wordlist.txt:W1,./domains.txt:W2
</code></pre></div></div>
<p>This would scan each of the domains in our <code class="language-plaintext highlighter-rouge">domains.txt</code> files using the wordlist from <code class="language-plaintext highlighter-rouge">wordlist.txt</code>, allowing us to run at scale without needing the use of outside scripting or applications.</p>
<p>The order of the wordlists control in what order the requests are sent. In clusterbomb mode (default) ffuf will iterate over the entire first wordlist before moving on to the second item in the second wordlist.</p>
<p>Why does this matter you wonder? Let me give you an example:</p>
<p>Lets say we have a wordlist with 1000 domains <code class="language-plaintext highlighter-rouge">domains.txt</code> and a wordlist with 1000 directories <code class="language-plaintext highlighter-rouge">wordlist.txt</code>.</p>
<p>If we run:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://FUZZDOMAIN/FUZZDIR -w ./wordlist.txt:FUZZDIR,./domains.txt:FUZZDOMAIN
</code></pre></div></div>
<p>ffuf will try every directory for the first domain, then every directory on the second domain.
When running with many threads, this means sending 1000 requests to the same server in a very short amount of time.
This often leads to getting rate-limited or banned.</p>
<p>If we on the other hand swap the order of the wordlists and run:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://FUZZDOMAIN/FUZZDIR -w ./domains.txt:FUZZDOMAIN,./wordlist.txt:FUZZDIR
</code></pre></div></div>
<p>ffuf will try the first directory on all domains, before moving on to the next directory and trying that on all domains.
This way you can send more requests without overloading the target servers.</p>
<h2 id="wordlist-parameter-bug">Wordlist Parameter Bug</h2>
<p>In older versions of FFUF there is a bug here whereby the <code class="language-plaintext highlighter-rouge">w</code> flag needs to be made use of multiple times for this to work as intended. If you receive the error:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Encountered error(s): 1 errors occurred.
* Keyword W1, defined, but not found in headers, method, URL or POST data.
</code></pre></div></div>
<p>Then you should instead either upgrade FFUF to the latest version, or use the <code class="language-plaintext highlighter-rouge">w</code> flag multiple times, like so:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://W2/W1 -w ./wordlist.txt:W1 -w ./domains.txt:W2
</code></pre></div></div>
<p>More information can be found here: https://github.com/ffuf/ffuf/issues/290</p>
<h1 id="handling-authentication">Handling Authentication</h1>
<h2 id="cookie-based-authentication">Cookie Based Authentication</h2>
<p>Often when performing a scan you will want to brute force behind an authentication point. In order to do this, FFUF provides the <code class="language-plaintext highlighter-rouge">b</code> flag for you to pas cookie data. These aren’t limited to authentication based cookies, and any area of the cookie (from names to values) can also be fuzzed with a wordlist for additional discovery.</p>
<h2 id="header-based-authentication">Header Based Authentication</h2>
<p>If authentication for the application is via HTTP header-based authentication then the <code class="language-plaintext highlighter-rouge">H</code> flag should be used. As with the <code class="language-plaintext highlighter-rouge">b</code> flag, this can be used to pass or fuzz any headers, and not just for passing required elements for authentication.</p>
<p>In addition to authentication, or fuzzing points, the <code class="language-plaintext highlighter-rouge">H</code> flag can also be utilised in situations where you’re required to “call your shot” by specifying a custom header for a client, or Bug Bounty engagement, so the defensive teams of those organisations can identify your traffic.</p>
<h2 id="more-complex-authentication-flows">More Complex Authentication Flows</h2>
<p>Occasionally, you’ll come accross authentication flows or fuzzing situations Burp Suite can’t provide. In those cases, I suggest creating an additional interface in Burp Suite and making use of Burp Suite Macros to acomplish this. Instructions for doing so can be found further on within this guide.</p>
<h1 id="threads">Threads</h1>
<p>By default FFUF will use 40 threads to execute. Essentially, this means that FFUF will start 40 seperate processes to execute the commands that you’ve provided. It may be tempting to set this much higher, but this will be limited by the power of your system, and the destination system you’re scanning against. If you’re in a network environment, such as HackTheBox, or OSCP then setting this higher may not pose much of an issue. If, however, you’re working on a production system over the internet then you are likely better off spending time tailoring the flags you’re passing to FFUF, and keeping your thread count lower, than trying to acheive a quicker result merely with raw thread count. Various flags you can use to better tailor your requests can be found further throughout this guide.</p>
<h1 id="using-silent-mode-for-passing-results">Using Silent Mode for Passing Results</h1>
<p>By default FFUF will strip colour from results (unless you enable it with the <code class="language-plaintext highlighter-rouge">-c</code> flag). This makes results easy to pass to other application, for additional work. One challenge here, is the header information, essentially:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
/'___\ /'___\ /'___\
/\ \__/ /\ \__/ __ __ /\ \__/
\ \ ,__\\ \ ,__\/\ \/\ \ \ \ ,__\
\ \ \_/ \ \ \_/\ \ \_\ \ \ \ \_/
\ \_\ \ \_\ \ \____/ \ \_\
\/_/ \/_/ \/___/ \/_/
v1.0.2
________________________________________________
:: Method : GET
:: URL : https://codingo.io/FUZZ
:: Follow redirects : false
:: Calibration : false
:: Timeout : 10
:: Threads : 40
:: Matcher : Response status: 200,204,301,302,307,401,403
________________________________________________
</code></pre></div></div>
<p>and the footer:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>:: Progress: [3/3] :: Job [1/1] :: 0 req/sec :: Duration: [0:00:10] :: Errors: 0 ::
</code></pre></div></div>
<p>To remove this, and only show results that line up with the <code class="language-plaintext highlighter-rouge">matcher</code> filters, you can use the silent flag, <code class="language-plaintext highlighter-rouge">-s</code>. This flag will enforce only successful hits to be shown. For example, our command from earlier, if exapnded with <code class="language-plaintext highlighter-rouge">-s</code> becomes:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -u https://codingo.io/FUZZ -w ./wordlist.txt -s
</code></pre></div></div>
<p>Which will then only show the result:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>admin
</code></pre></div></div>
<p>As that responds with a <code class="language-plaintext highlighter-rouge">301</code> request, which is within our <code class="language-plaintext highlighter-rouge">matcher</code> filters.</p>
<h1 id="error-handling-and-scan-tweaks">Error Handling and Scan Tweaks</h1>
<h2 id="automatically-calibrate-filtering">Automatically Calibrate Filtering</h2>
<p>The <code class="language-plaintext highlighter-rouge">ac</code> flag in FFUF can be used to automatically calibrate filtering of requests. This flag tells FFUF to send a number of preflight checks before brute forcing begins and to quantify common elements of those requests for further filtering. For example, FFUF may send random strings, and if each of those responses were a <code class="language-plaintext highlighter-rouge">200</code> response code, with a common content length, then that content length would be automatically filtered from future results.</p>
<h2 id="custom-automatic-calibration-filtering">Custom Automatic Calibration Filtering</h2>
<p>In addition to the <code class="language-plaintext highlighter-rouge">ac</code> flag, FFUF allows you to provide the seed request to build autocalibration against, instead of using pre-flight checks. A good example where this shines, is with Virtual Host Scanning. When checking for Virtual Hosts (VHosts), you are seeking responses that don’t match the host request. If we were to do that with <code class="language-plaintext highlighter-rouge">acc</code>, we could use the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -w vhostnames.txt -u https://target -H "Host: FUZZ. target" -acc "www"
</code></pre></div></div>
<p>This would send a preflight check to our target to capture the content-length and response code of <code class="language-plaintext highlighter-rouge">www</code>, and then highlight only responses which have a different content length that return from our wordlist. This greatly helps to eliminate false positives, and in these types of cases is more accurate than <code class="language-plaintext highlighter-rouge">ac</code> which would use random strings to capture the response, and is unlikely to be as accurate for this type of (and other types of) fuzzing activity.</p>
<h2 id="immediately-stop-on-error-case">Immediately Stop on Error Case</h2>
<p>FFUF can force stop once an error is received, using the <code class="language-plaintext highlighter-rouge">sa</code> flag. This overrules any of the other error condition flags (<code class="language-plaintext highlighter-rouge">se</code> and <code class="language-plaintext highlighter-rouge">sf</code>) as their thresholds would never be met.</p>
<h2 id="stop-on-spurious-errors">Stop on Spurious Errors</h2>
<p>FFUF has a flag to allow jobs to automatically stop when a percentage of the recent requests have thrown an error. This flag is <code class="language-plaintext highlighter-rouge">se</code>, and will end the job if the last 50 requests have thrown a <code class="language-plaintext highlighter-rouge">403</code> response code 95% of the time, or if 20% of the responses have been a <code class="language-plaintext highlighter-rouge">429</code> response code. This can be safer than the flag <code class="language-plaintext highlighter-rouge">sf</code>, which will only stop if a portion of the entire request pool have errored, and not just recent requests. These limits may change over time, and are referenced here, should you wish to manually review them: https://github.com/ffuf/ffuf/blob/master/pkg/ffuf/job.go#L382-L407</p>
<h1 id="request-throttling-and-delays">Request Throttling and Delays</h1>
<p>With production hosts, or under various testing conditions you will need to throttle your responses. When these conditions are required, FFUF provides a number of options.</p>
<h2 id="delay-between-requests">Delay Between Requests</h2>
<p>The <code class="language-plaintext highlighter-rouge">p</code> flag specifies the seconds of delay between requests. This can be a range, or a set figure. For example, a <code class="language-plaintext highlighter-rouge">p</code> value of <code class="language-plaintext highlighter-rouge">2</code> would enforce a 2 second delay between each request, whilst a <code class="language-plaintext highlighter-rouge">p</code> value of <code class="language-plaintext highlighter-rouge">0.1-2.0</code> would enforce a random delay between 0.1-2 seconds.</p>
<h2 id="limited-max-requestssecond">Limited Max Requests/second</h2>
<p>As FFUF is a multi threaded application, you can easily end up overwealming a destination target with too many requests. In order to control this, you can specify a maximum number of requests that can be sent per second. This can be set with the <code class="language-plaintext highlighter-rouge">-rate</code> flag. For example, if you wish to limit to 2 requests per second, then you would specify <code class="language-plaintext highlighter-rouge">-rate 2</code>.</p>
<h1 id="match-options">Match Options</h1>
<h2 id="match-on-response-code">Match on Response Code</h2>
<p>There are a variety of predescribed matching options in FFUF, the most common one that you’ll find yourself using is <code class="language-plaintext highlighter-rouge">mc</code>, which matches the response code. In many cases, you’ll want to change this to limit to only <code class="language-plaintext highlighter-rouge">200</code> requests, to help isolate the results to content that you’re seeking.</p>
<h2 id="match-on-regular-expression">Match on Regular Expression</h2>
<p>In some cases, however, you may be fuzzing for more complex bugs and want to filter based on a <a href="https://en.wikipedia.org/wiki/Regular_expression">regular expression</a>. For example, if you’re filtering for a path traversal bug you may wish to pass a value of <code class="language-plaintext highlighter-rouge">-mr "root:"</code> to FFUF to only identify successful responses that indicate a successful retreival of <code class="language-plaintext highlighter-rouge">/etc/passwd</code>. Such cases are quite common, and highlight some of the power that FFUF brings into fuzzing that competitive offerings are not yet able to match.</p>
<h1 id="filter-and-matches">Filter and Matches</h1>
<p>As useful as matches are, filters being the inverse of matches can be just as, if not more useful. When returning the results of a page that has a sink (a location where your source, or wordlist item is reflected in the page) within the response, it can be more useful to filter the number of words in a page, rather than filter by content length. For this purpose, FFUF provides <code class="language-plaintext highlighter-rouge">fw</code>, or filter words. If you can identify the number of words commonly in the response, you can apply this filter to remove any results that have your content length. If words aren’t specific enough, you can also filter on the number of lines within the HTTP response, using <code class="language-plaintext highlighter-rouge">fl</code>.</p>
<p>Much like filters, you can also filter based on content length (<code class="language-plaintext highlighter-rouge">fc</code>) to remove response types from the results. This can be especially useful where you want to first filter for all defaults, which includes the <code class="language-plaintext highlighter-rouge">301</code> response code, and then filter this response code out from the results to see more specific responses.</p>
<h1 id="sending-ffuf-scans-via-burp-suite">Sending FFUF scans via Burp Suite</h1>
<p>For a variety of reasons, you’ll often find yourself wanting your FFUF scans to be sent via Burp Suite. Notably, there’s a few ways to acomplish this goal, and it’s important to understand each of them, and apply the right one for your use case.</p>
<h2 id="locally-using-replay-proxy">Locally, Using Replay Proxy</h2>
<p>FFUF has a command within it, <code class="language-plaintext highlighter-rouge">replay-proxy</code> to dictate. This will retoute successful commands (ones that hit your matches, and not your filters) to Burp Suite proxy for further investigation. Notably, this does mean that you’re doubling your requests, and this should be used in situations where it makes sense to do so.</p>
<p>If for whatever reason (such as engagement terms) you need to send all information via Burp Suite, and not just successful traffic, then you can instead use <code class="language-plaintext highlighter-rouge">x</code> which will replay all requests via a Burp Suite project, regardless of whether they line up with FFUF filters/matches or not.</p>
<h2 id="using-an-interface">Using an Interface</h2>
<p>Occasionally, you’ll encounter situations where you need all of your FFUF (or another tools) traffic to be send via Burp Suite over a Burp Suite Interface. This could be due to engagement logging (required by the firm you’re testing for/against), or due to a complex authentication schema that Burp Suite is better positioned to handle. Personally, I’ve also found this useful for fuzzing various elements (such as CSRF tokens) in conjunction with Burp Suite Macros. Whatever the use case, the method for doing this is quite simple. Firstly, we need to go to Burp Suite and setup a second interface, you can do this under <em>proxy</em>-><em>options</em>-><em>add</em></p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/burp-add-second-interface.png" alt="FFUF" /></p>
<p>Under binding, set a port, for the second interface I prefer to use <code class="language-plaintext highlighter-rouge">8181</code> (as <code class="language-plaintext highlighter-rouge">8080</code> is the default and I find this easy to recall).</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/burp-interface-binding.png" alt="FFUF" /></p>
<p>Under the request handling flag, set “Redirect to host” and “Redirect to port” to match that of our destination target:</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/burp-interface-redirections.png" alt="FFUF" /></p>
<p>After we’ve done that, leave other settings the same and click <em>ok</em>. We can then target <code class="language-plaintext highlighter-rouge">https://127.0.0.1:8181</code> with any of our tools, including FFUF, and it will automatically redirect to the destination target. This means, instead of using <code class="language-plaintext highlighter-rouge">http://target.com/path/FUZZ</code> in FFUF to focus on our target, we can use <code class="language-plaintext highlighter-rouge">https://127.0.0.1:8181/path/FUZZ</code>. Everything will work as it did before, except the requests are being sent to, and out from Burp Suite.</p>
<p>Be cautious when using this approach on large wordlists, as Burp Suite will store the history within your associated project, and passing large fuzzes via Burp Suite is likely to cause your project file to become bloated, and unwieldy quickly.</p>
<h2 id="remote-vps-traffic-via-a-reply-proxy">Remote VPS Traffic via a Reply Proxy</h2>
<p>When using a remote VPS you’ll occasionally hit decisions in your testing that would be aided by using a local version of Burp Suite. To help aid in this, when fuzzing with FFUF you can open a reverse SSH tunnel and combine it with <code class="language-plaintext highlighter-rouge">reply-proxy</code> on your remote VPS to replay it over the remote port, to your local Burp Suite instance.</p>
<p>First connect to your remote VPS over SSH server using:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ssh -R 8888:localhost:8080 user@remotevps
</code></pre></div></div>
<p>And then run FFUF with the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/go/bin/ffuf -u http://codingo.io/FUZZ -w ./wordlist -replay-proxy http://127.0.0.1:8888
</code></pre></div></div>
<p>Since we bound port 8888 to relay over our reverse SSH tunnel to our remote burp instance, on port 8080, this will then replay back in Burp Suite.</p>
<h1 id="advanced-wordlist-usage">Advanced Wordlist Usage</h1>
<p>When using multiple wordlists, FFUF has two modes of operation. The first, and the default, is clusterbomb. This takes both wordlists and tries all possible combinations of them, and is best for brute forcing operations. By default FFUF will use the clusterbomb attack mode, however you can specify other modes (for now, just pitchfork and clusterbomb) using the <code class="language-plaintext highlighter-rouge">mode</code> flag.</p>
<p>For example, let’s assume we had a wordlist called “users”, with two users, “codingo” and “anonymous”. In addition, we’ll assume we have a wordlist “passwords”, with two items, “hunter1”, and “password”. In clusterbomb mode, all combinations of these would be tried, resulting in the following output.</p>
<p>Alternatively, FFUF provides another mode called “pitchfork”. This mode is intended for when you want to use wordlists in series. For example, let’s assume that you have a list of passwords, that go with a list of users and want to fuzz via a username and parameter endpoint. In this example, the password “hunter1”, would be tried with the user “codingo”, and the password “password” would be tried with the user “anoymous”, however that would be the end of the operation, and further combinations would not be tried.</p>
<p>Each has its own use cases, and it’s important to know how to use both, however if you’re unsure which to use, it’s best to stick with the default, clusterbomb.</p>
<h2 id="clusterbomb">Clusterbomb</h2>
<p>Most useful for a brute-force attack style, the clusterbomb will try all combinations of payloads. As Burp Suite Intruder operates with the same kind of wordlist approaches, I’ve found this is best explained by Sjord, <a href="https://www.sjoerdlangkemper.nl/2017/08/02/burp-intruder-attack-types/">here</a>. To paraphrase Sjord, essentially the clusterbomb tries all possible combinations, while still keeping the first payload set in the first position and the second payload set in the second position. As shown in the following example:</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/clusterbomb-attack.png" alt="FFUF" /></p>
<p>Here we can see that the first payload position is used in position one, <code class="language-plaintext highlighter-rouge">456</code>. And the second, in postion two, <code class="language-plaintext highlighter-rouge"><br</code>. The first payload is then rotated, whilst the second isn’t, until the first list has been exhausted at which time the second list continues through the same operation. Operating in this style ensures that all possible permutations are tested.</p>
<h2 id="pitchfork">Pitchfork</h2>
<p>Much like the Clusterbomb approach, I’ve found the Pitchfork style of fuzzing is also best explained by Sjord, <a href="https://www.sjoerdlangkemper.nl/2017/08/02/burp-intruder-attack-types/">here</a>. To paraphase, the pitchfork attack type uses one payload set for each position. It places the first payload in the first position, the second payload in the second position, and so on. This attack type is useful if you have data items that belong together. For example, you have usernames with corresponding passwords and want to know whether they work with this web application.</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/pitchfork-attack.png" alt="FFUF" /></p>
<p>As you can see when compared to the clusterbomb atack, the pitchfork attack works the wordlists in series. Not all combinations will be reached, but the use case for these is that they aren’t intended to and doing so would be a waste of requests.</p>
<h1 id="handling-output">Handling Output</h1>
<h2 id="html-output">HTML Output</h2>
<h2 id="using-silent-and-tee">Using Silent and Tee</h2>
<p>If you want to print results only, without all of the padding, the <code class="language-plaintext highlighter-rouge">s</code> flag, or silent mode, works great for this. For example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -request /tmp/request.txt -w ./wordlist.txt -s
</code></pre></div></div>
<p>With our original example, will only output <code class="language-plaintext highlighter-rouge">admin</code>, as it’s the only successful match. This can also be useful to pass to other tools, however when doing so I suggest also using <a href="https://en.wikipedia.org/wiki/Tee_(command)"><code class="language-plaintext highlighter-rouge">tee</code></a>. The <code class="language-plaintext highlighter-rouge">tee</code> command will output the results to console, whilst also redirecting it as <code class="language-plaintext highlighter-rouge">stdout</code>, allowing other applications to consume it. For example, the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -request /tmp/request.txt -w ./wordlist.txt -s | tee ./output.txt
</code></pre></div></div>
<p>Would output to the console and write to output.txt. This is a useful trick for a number of tools, including those that don’t stream output, to allow you to see results in realtime, whilst also streaming them to a file.</p>
<h1 id="importing-requests">Importing Requests</h1>
<p>On of the easiest ways to work with complex queries is to simply save the request you’re working with from your intercepting proxy (such as Burp Suite), set your fuzzing paths, and then import it into FFUF for usage. You can do this with the <code class="language-plaintext highlighter-rouge">request</code> flag in FFUF, as explained below.</p>
<p>Going back to our original fuzzing example, let’s assume we visited codingo.io in Burp Suite, and we captured the following request:</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/request-to-codingo.png" alt="FFUF" /></p>
<p>We can right click in the request, and select <code class="language-plaintext highlighter-rouge">Copy to File</code>:</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/burp-suite-save-request.png" alt="FFUF" /></p>
<p>Be sure <em>not</em> to select <code class="language-plaintext highlighter-rouge">Save item</code> as that will save this in a format known only to Burp Suite, and not of use to FFUF.</p>
<p>Once we’ve saved the file, we then need to open it in our favorite editor, and add our fuzzing points. For this example, I want to brute force items at the top level of the <code class="language-plaintext highlighter-rouge">codingo.io</code> domain and so I’m adding <code class="language-plaintext highlighter-rouge">FUZZ</code> on the top line, as shown below:</p>
<p><img src="http://codingo.io/assets/2020-09-17-ffuf/request-editing-vim.png" alt="FFUF" /></p>
<p>We can then open our request in FFUF, and instead of passing cookie information or a URL, we can use <code class="language-plaintext highlighter-rouge">request</code> to feed it the information in our saved request. In this case, this would look like the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffuf -request /tmp/request.txt -w ./wordlist.txt
</code></pre></div></div>
<h1 id="contributing-to-this-guide">Contributing to this guide</h1>
<p>This guide is open source, maintained on Github. If you’d like to contribute to this guide, or to make a correction, you can do so here: https://github.com/codingo/codingo.github.io</p>
<h2 id="contributors">Contributors</h2>
<p>The following authors have contributed to this guide:</p>
<table>
<thead>
<tr>
<th>Date</th>
<th>Contributor</th>
<th>Twitter</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>17 Sep 2020</td>
<td>codingo</td>
<td><a href="https://twitter.com/codingo_">https://twitter.com/codingo_</a></td>
<td>Initial Draft / Publication</td>
</tr>
<tr>
<td>28 Sep 2020</td>
<td>p4fg</td>
<td>N/A</td>
<td>Added additional hints on fuzzing multiple domains</td>
</tr>
</tbody>
</table>Summary This guide is a large summary of the information security tool, FFUF. This is also paired with a video companion guide, shown below:Decompiling C# by Example with Cracknet2017-07-25T07:01:58+00:002017-07-25T07:01:58+00:00http://codingo.io/reverse-engineering/ctf/2017/07/25/Decompiling-CSharp-By-Example-with-Cracknet<h1 id="what-is-cracknet">What is Cracknet?</h1>
<p>As a part of the SecTalks May CTF I built a .Net reverse engineering challenge, Cracknet. I’ve since made this available on Github, <a href="https://github.com/codingo/cracknet">here</a>.</p>
<p>Although it’s possible to complete this challenge by bypassing a JMP instruction in assembly the intention of this challenge was to introduce participants to decompiling .Net applications by patching the application.</p>
<h1 id="exploring-cracknet-functionality">Exploring CrackNet functionality</h1>
<p>When you first open CrackNet you’re presented with the following:</p>
<p><img src="http://codingo.io/assets/2017-07-25-CrackNet/1-CrackNet-FirstLoad.PNG" alt="CrackNet First Load" /></p>
<p>When you enter a guess you’ll be presented with either the flag, or told it’s incorrect and then presented with a countdown until you can make another guess:</p>
<p><img src="http://codingo.io/assets/2017-07-25-CrackNet/2-CrackNet-GuessCounter.PNG" alt="CrackNet Guess Counter" /></p>
<p>This wait time eliminates the ability to brute force the flag and after five incorrect guesses you’ll hear a quick mario tune (if system speaker is enabled) and be presented with the following:</p>
<p><img src="http://codingo.io/assets/2017-07-25-CrackNet/3-Cracknet-GameOver.PNG" alt="CrackNet GameOver" /></p>
<h1 id="decompiling-with-dnspy">Decompiling with dnSpy</h1>
<h2 id="why-and-what-is-dnspy">Why and what is dnSpy?</h2>
<p>C# is a “jittable” langauge in that it’s not compiled purely down to machine code like C is, however it’s also not completely interpretted like Python is. C# holds the middle ground and source code written is compiled into an intermediate language (IL) according to the CLI specification. When a C# program is executed, the assembly is loaded into the CLR, then, if all security requirements are met, the CLR performs just in time (JIT) compilation to convert the IL code to native machine instructions.</p>
<p>This brings in a fantistic Github project - <a href="https://github.com/0xd4d/dnSpy">dnSpy</a>. <a href="https://github.com/0xd4d/dnSpy">dnSpy</a> is a tool to reverse engineer .NET assemblies from their CLI state back to somewhat interprettable and editable code. It includes a decompiler, a debugger and an assembly editor that allows you to modify or debug .Net applications as required. It’s important to note that this code isn’t perfect as it’s a representation post-compilation from Visual Studio. Redundant blocks in many cases won’t be present and program flow and variables will be represented differently. To get a full picture of this you can compare the decompiled version of Cracknet with the source code hosted on Github.</p>
<h2 id="loading-the-project">Loading the project</h2>
<p>After cloning the dnSpy repository and opening the project you will be presented with something similar to the following:</p>
<p><img src="http://codingo.io/assets/2017-07-25-CrackNet/4-dnSpy.PNG" alt="dnSpy initial load" /></p>
<h2 id="navigate-to-main">Navigate to main</h2>
<p>We then want to navigate to the main entry point of our application so we can understand what’s happening:</p>
<p><img src="http://codingo.io/assets/2017-07-25-CrackNet/5-DecompileMain.PNG" alt="dnSpy Decompile Main" /></p>
<h2 id="patching-the-application">Patching the application</h2>
<p>Looking at the code we can see that the instruction for decrypting the flag can only be reached by entering the result of the decryption (the flag). At this point we could take the Crypto class in this project along with the AES key and create a new application for our flag reveal however it’s far more ideal if we instead patch our binary to bypass the if conditional and show the flag early. To do this we first need to select <strong>Edit Method</strong> in dnSpy (found in the right click menu):</p>
<p><img src="http://codingo.io/assets/2017-07-25-CrackNet/6-EditMethod.png" alt="dnSpy Decompile Main" /></p>
<p>We can then relocate our key decrytion to the beginning of the method and comment out the remaining code. If we only move the key decryption and don’t comment out the remaining code our key will be overwritten as the screen is redrawn for the timer.</p>
<p><img src="http://codingo.io/assets/2017-07-25-CrackNet/7-PatchedCode.PNG" alt="dnSpy Decompile Main" /></p>
<p>The relevant patched code is as follows:</p>
<figure class="highlight"><pre><code class="language-csharp" data-lang="csharp"><span class="k">static</span> <span class="k">void</span> <span class="nf">Main</span><span class="p">(</span><span class="kt">string</span><span class="p">[]</span> <span class="n">args</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">Debug</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">"flag{Not a real flag. Strings would be too easy"</span><span class="p">);</span>
<span class="n">Program</span><span class="p">.</span><span class="nf">PrintBanner</span><span class="p">();</span>
<span class="c1">//int guesses = 5;</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="kt">string</span><span class="p">.</span><span class="nf">Format</span><span class="p">(</span><span class="s">"Success! Flag: {0}!"</span><span class="p">,</span> <span class="n">Crypto</span><span class="p">.</span><span class="nf">DecryptStringAES</span><span class="p">(</span><span class="s">"EAAAAB+ljfnegBraKanx/SJLBfrGhIDfffz8MOc922hrm0aK44KwgXmu9GHrIU+LjyBwmQ=="</span><span class="p">)));</span>
<span class="cm">/*
while (true)
{
bool flag = guesses < 1;
if (flag)
{
Program.PrintGameOver();
}
Program.PrintTimer(3);
Program.PrintGuesses(guesses);
Console.Write("Enter password: ");
string input = Console.ReadLine();
string password = Crypto.DecryptStringAES("EAAAAOkz8XiBpPhe0j3CnxGt4D5Qb0H2vh9/IeXrt1w4r313");
bool flag2 = input != null && input.ToLower().Equals(password);
if (flag2)
{
Console.WriteLine(string.Format("Success! Flag: {0}!", Crypto.DecryptStringAES("EAAAAB+ljfnegBraKanx/SJLBfrGhIDfffz8MOc922hrm0aK44KwgXmu9GHrIU+LjyBwmQ==")));
Program.StarWars();
Environment.Exit(0);
}
int num = guesses;
guesses = num - 1;
Console.WriteLine("Incorrect! Please wait to try again.");
Console.Beep(350, 250);
Console.Beep(300, 500);
}
*/</span>
<span class="p">..</span> <span class="n">remainder</span> <span class="n">redacted</span> <span class="p">..</span></code></pre></figure>
<h2 id="debug-and-reveal-the-flag">Debug and reveal the flag</h2>
<p>We then run our patched application within dnSpy to reveal the flag:</p>
<p><img src="http://codingo.io/assets/2017-07-25-CrackNet/9-FlagReveal.PNG" alt="dnSpy Decompile Main" /></p>
<p>And wala!</p>What is Cracknet? As a part of the SecTalks May CTF I built a .Net reverse engineering challenge, Cracknet. I’ve since made this available on Github, here.Understanding Integer Overflows - CrikeyCon 2017, Impossible Math Writeup2017-03-08T07:10:58+00:002017-03-08T07:10:58+00:00http://codingo.io/ctf/writeup/programming/crikeycon/tutorial/2017/03/08/CTF-Writeup-CrikeyCon-2017-ImpossibleMath<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Category: Coding
Points: 400
Solves: 7
Description: ctf.crikeycon.com:43981
</code></pre></div></div>
<h1 id="enumeration">Enumeration</h1>
<p>Before doing anything else on the host since we were provided with an unual port and address I attempted to ncat to it, receiving the following:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/ImpossibleMath-Preview.png" alt="Impossible Math Preview" /></p>
<h1 id="identifying-the-core-problem">Identifying the core problem</h1>
<p>Since the math is impossible there’s likely a trick here. With that in mind I figure we need to overload an operator (integer overflow) and try passing a large number as input:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/OverflowCheck.png" alt="Impossible Math Overflow Check" /></p>
<p>Awesome! Our integer overloads by wrapping around. To gather a bit more information I also tried an integer underflow:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/UnderflowCheck.png" alt="Impossible Math Underflow Check" /></p>
<p>The same result. I now had to identify the figure we wrap around. These numbers are a bit irritatingly long to work with so I tried something a bit smaller to see if I could something more manageable:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/ManageableNumbers.png" alt="Impossible Math Manageable Numbers" /></p>
<p>We can use the following to identify our overflow point:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/OverflowFormula.png" alt="Impossible Math Manageable Numbers" /></p>
<p>4294967296 is exactly 2^32, which is 1 beyond the maximum supported by unsigned int (32 bits), further supporting our case that this is an integer overflow exercise.</p>
<p>To validate this, I should be able to pass this value to any problem and receive 0 back as a response (as it will reach the signed amount and loop back once), as follows:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/ZeroTest.png" alt="Impossible Math Manageable Numbers" /></p>
<p>For the remainder of this exercise I’m going to refer to the variables from our second-last screenshot as the following:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/Variables.png" alt="Impossible Math Manageable Numbers" /></p>
<p>Since our number wraparounds we now know we need a number with the following conditions:</p>
<ul>
<li>Our overflow must be lower than 4294967296 but higher than our destination to pass the first condition.</li>
<li>Our overflow needs to exceed 4294967296 when multiplied by the multiplier and result in the destination</li>
</ul>
<h1 id="calculating-the-correct-overflow">Calculating the correct overflow</h1>
<p>As a reminder, we can calculate our overflow using the formula from earlier:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/VerboseVariables.png" alt="Impossible Math Formula" /></p>
<p>I turned this into a proof of concept by generating a new ncat session, which asked the following:</p>
<figure class="highlight"><pre><code class="language-text" data-lang="text">Solve for X, where:
X > 37864
X * 8 = 37864</code></pre></figure>
<p>To generate our answer, I used the following:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">python</span> <span class="o">-</span><span class="n">c</span> <span class="s">'print((2**32+destination)/multiplier)'</span></code></pre></figure>
<p>This generated the answer of 536875645 as follows:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/FirstAnswer.png" alt="Impossible Math First Answer" /></p>
<p>Under some circumstances, we would then be able to pipe our answer into a new ncat session but since our variables change on each connection we need to do this manually to verify it’s correct:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/FirstFlag.png" alt="Impossible Math First Flag" /></p>
<p>Success! Our flag was revealed.</p>
<h1 id="creating-an-automatic-answer-tool">Creating an automatic answer tool</h1>
<p>Manual answers are great, but this is classified as a coding challenge, not a mathematical one!</p>
<p>There are a few core processes to this part of the exercise. First is forming an open connection and identifying our destination and multiplier from the data that comes back. We’ve also been asked in the banner of our connection to limit our calculations to 4 bytes, so we’ll make sure we limit what we request at a time.</p>
<p>Splitting our variables out of the calculation is quite easy. They’re both present on the line reflected as:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/SecondCheck.png" alt="Impossible Math Second Check" /></p>
<p>We can use regular expressions to identify this packet stream from the equals sign, and then split our values out into capture groups using another expression. Putting this boilerplate together looks like the following:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c1">#!/usr/bin/python3
</span>
<span class="kn">import</span> <span class="nn">socket</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">import</span> <span class="nn">operator</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="n">MAXBUF</span> <span class="o">=</span> <span class="mi">4096</span>
<span class="n">SENTINEL</span> <span class="o">=</span> <span class="s">'flag'</span>
<span class="n">CTF_BOT</span> <span class="o">=</span> <span class="p">(</span><span class="s">'ctf.crikeycon.com'</span><span class="p">,</span> <span class="mi">43981</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span>
<span class="n">client</span> <span class="o">=</span> <span class="n">socket</span><span class="p">.</span><span class="n">socket</span><span class="p">(</span><span class="n">socket</span><span class="p">.</span><span class="n">AF_INET</span><span class="p">,</span> <span class="n">socket</span><span class="p">.</span><span class="n">SOCK_STREAM</span><span class="p">)</span>
<span class="n">client</span><span class="p">.</span><span class="n">connect</span><span class="p">(</span><span class="n">CTF_BOT</span><span class="p">)</span>
<span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="n">data</span> <span class="o">=</span> <span class="sa">b</span><span class="s">''</span>
<span class="c1"># receive and store data
</span> <span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="n">chunk</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">recv</span><span class="p">(</span><span class="n">MAXBUF</span><span class="p">)</span>
<span class="n">data</span> <span class="o">+=</span> <span class="n">chunk</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">chunk</span><span class="p">)</span> <span class="o"><</span> <span class="n">MAXBUF</span><span class="p">:</span>
<span class="k">break</span>
<span class="c1"># store decoded data for future usage
</span> <span class="n">decoded</span> <span class="o">=</span> <span class="n">data</span><span class="p">.</span><span class="n">decode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span>
<span class="c1"># print out response packet
</span> <span class="k">print</span><span class="p">(</span><span class="n">decoded</span><span class="p">)</span>
<span class="c1"># our flag contains flag{}, once it's revealed print received data and exit
</span> <span class="k">if</span> <span class="n">SENTINEL</span> <span class="ow">in</span> <span class="n">decoded</span><span class="p">:</span>
<span class="k">break</span>
<span class="c1"># skip loop until we see our X * Y = Z line
</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="s">'[=]'</span><span class="p">,</span> <span class="n">decoded</span><span class="p">):</span>
<span class="k">continue</span>
<span class="c1"># select integers and store into capture groups
</span> <span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="s">'(\d+) = (\d+)'</span><span class="p">,</span> <span class="n">decoded</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="s">'multiplier: '</span> <span class="o">+</span> <span class="n">match</span><span class="p">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
<span class="k">print</span><span class="p">(</span><span class="s">'destination: '</span> <span class="o">+</span> <span class="n">match</span><span class="p">.</span><span class="n">group</span><span class="p">(</span><span class="mi">2</span><span class="p">))</span></code></pre></figure>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/BoilerPlateExec.png" alt="Impossible Math Boilerplate Execution" /></p>
<p>Great! We now have what we need in a variable. Referring back to our formula above, we now need to calculate:</p>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/VerboseVariables.png" alt="Impossible Math Formula" /></p>
<p>This will look like the following (note that we cast our regular expressions back to integers to prevent operand exceptions):</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">multiplier</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">match</span><span class="p">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
<span class="n">destination</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">match</span><span class="p">.</span><span class="n">group</span><span class="p">(</span><span class="mi">2</span><span class="p">))</span>
<span class="n">overflow</span> <span class="o">=</span> <span class="nb">int</span><span class="p">((</span><span class="mi">2</span><span class="o">**</span><span class="mi">32</span><span class="o">+</span><span class="n">destination</span><span class="p">)</span> <span class="o">/</span> <span class="n">multiplier</span><span class="p">)</span></code></pre></figure>
<h1 id="final-script">Final Script</h1>
<p>Putting it all together we receive the following:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c1">#!/usr/bin/python3
</span>
<span class="kn">import</span> <span class="nn">socket</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">import</span> <span class="nn">operator</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="n">MAXBUF</span> <span class="o">=</span> <span class="mi">4096</span>
<span class="n">SENTINEL</span> <span class="o">=</span> <span class="s">'flag'</span>
<span class="n">CTF_BOT</span> <span class="o">=</span> <span class="p">(</span><span class="s">'ctf.crikeycon.com'</span><span class="p">,</span> <span class="mi">43981</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span>
<span class="n">client</span> <span class="o">=</span> <span class="n">socket</span><span class="p">.</span><span class="n">socket</span><span class="p">(</span><span class="n">socket</span><span class="p">.</span><span class="n">AF_INET</span><span class="p">,</span> <span class="n">socket</span><span class="p">.</span><span class="n">SOCK_STREAM</span><span class="p">)</span>
<span class="n">client</span><span class="p">.</span><span class="n">connect</span><span class="p">(</span><span class="n">CTF_BOT</span><span class="p">)</span>
<span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="n">data</span> <span class="o">=</span> <span class="sa">b</span><span class="s">''</span>
<span class="c1"># receive and store data
</span> <span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="n">chunk</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">recv</span><span class="p">(</span><span class="n">MAXBUF</span><span class="p">)</span>
<span class="n">data</span> <span class="o">+=</span> <span class="n">chunk</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">chunk</span><span class="p">)</span> <span class="o"><</span> <span class="n">MAXBUF</span><span class="p">:</span>
<span class="k">break</span>
<span class="c1"># store decoded data for future usage
</span> <span class="n">decoded</span> <span class="o">=</span> <span class="n">data</span><span class="p">.</span><span class="n">decode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span>
<span class="c1"># print out response packet
</span> <span class="k">print</span><span class="p">(</span><span class="n">decoded</span><span class="p">)</span>
<span class="c1"># our flag contains flag{}, once it's revealed print recevied data and exit
</span> <span class="k">if</span> <span class="n">SENTINEL</span> <span class="ow">in</span> <span class="n">decoded</span><span class="p">:</span>
<span class="k">break</span>
<span class="c1"># skip loop until we see our X * Y = Z line
</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="s">'[=]'</span><span class="p">,</span> <span class="n">decoded</span><span class="p">):</span>
<span class="k">continue</span>
<span class="c1"># select integers and store into capture groups
</span> <span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="s">'(\d+) = (\d+)'</span><span class="p">,</span> <span class="n">decoded</span><span class="p">)</span>
<span class="n">multiplier</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">match</span><span class="p">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
<span class="n">destination</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">match</span><span class="p">.</span><span class="n">group</span><span class="p">(</span><span class="mi">2</span><span class="p">))</span>
<span class="n">overflow</span> <span class="o">=</span> <span class="nb">int</span><span class="p">((</span><span class="mi">2</span><span class="o">**</span><span class="mi">32</span><span class="o">+</span><span class="n">destination</span><span class="p">)</span> <span class="o">/</span> <span class="n">multiplier</span><span class="p">)</span>
<span class="c1"># encode and transfer
</span> <span class="n">client</span><span class="p">.</span><span class="n">send</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">overflow</span><span class="p">).</span><span class="n">encode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span><span class="o">+</span> <span class="sa">b</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">)</span></code></pre></figure>
<p><img src="http://codingo.io/assets/2017-03-08-Impossible-Math-Writeup/flag.png" alt="Impossible Math Flag" /></p>
<p>Success!</p>
<p>I hope this helped you to better understand integer overflows. If you’re Brisbane based, or find yourself here be sure to check out <a href="https://www.meetup.com/en-AU/SecTalks-Brisbane/">SecTalks</a>.</p>Category: Coding Points: 400 Solves: 7 Description: ctf.crikeycon.com:43981CrikeyCon 2017 - Fast Math Writeup2017-03-08T07:01:58+00:002017-03-08T07:01:58+00:00http://codingo.io/ctf/writeup/programming/crikeycon/2017/03/08/crikeycon-2017-fastmath<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Category: Coding
Points: 300
Solves: 14
Description: crikeyconctf.dook.biz:23776
</code></pre></div></div>
<h1 id="enumeration">Enumeration</h1>
<p>Before doing anything else on this host I attempted to connect to it, receiving the following:</p>
<p><img src="http://codingo.io/assets/2017-03-08-FastMath-Writeup/FastMath-Preview.png" alt="Math Preview" /></p>
<p>The time between being presented with the challenge and receiving a timeout was a mere two seconds. Although handy with a calculator this wouldn’t be possible without a script/bot. I also noted that the response and timing to answer didn’t change on a second connection, but the base operator did. I then felt comfortable writing a script to connect to the host and return an answer to basic math questions (+-/*).</p>
<h1 id="first-answer-script-attempt">First Answer Script Attempt</h1>
<p>I put together a bot which would connect to the host and parse the challenge question using regex so it could answer it using an expression. This resulted in the following:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c1">#!/usr/bin/python3
</span>
<span class="kn">import</span> <span class="nn">socket</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">import</span> <span class="nn">operator</span>
<span class="n">MAXBUF</span> <span class="o">=</span> <span class="mi">4096</span>
<span class="n">SENTINEL</span> <span class="o">=</span> <span class="s">'flag'</span>
<span class="n">CTF_BOT</span> <span class="o">=</span> <span class="p">(</span><span class="s">'crikeyconctf.dook.biz'</span><span class="p">,</span> <span class="mi">23776</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span>
<span class="n">client</span> <span class="o">=</span> <span class="n">socket</span><span class="p">.</span><span class="n">socket</span><span class="p">(</span><span class="n">socket</span><span class="p">.</span><span class="n">AF_INET</span><span class="p">,</span> <span class="n">socket</span><span class="p">.</span><span class="n">SOCK_STREAM</span><span class="p">)</span>
<span class="n">client</span><span class="p">.</span><span class="n">connect</span><span class="p">(</span><span class="n">CTF_BOT</span><span class="p">)</span>
<span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="n">data</span> <span class="o">=</span> <span class="sa">b</span><span class="s">''</span>
<span class="c1"># receive and store data
</span> <span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="n">chunk</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="n">recv</span><span class="p">(</span><span class="n">MAXBUF</span><span class="p">)</span>
<span class="n">data</span> <span class="o">+=</span> <span class="n">chunk</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">chunk</span><span class="p">)</span> <span class="o"><</span> <span class="n">MAXBUF</span><span class="p">:</span>
<span class="k">break</span>
<span class="c1"># store decoded data for future usage
</span> <span class="n">decoded</span> <span class="o">=</span> <span class="n">data</span><span class="p">.</span><span class="n">decode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span>
<span class="c1">#temporary
</span> <span class="k">print</span><span class="p">(</span><span class="n">decoded</span><span class="p">)</span>
<span class="c1">#
</span>
<span class="c1"># our flag likely contains flag{}, once it's revealed print received data and exit
</span> <span class="k">if</span> <span class="n">SENTINEL</span> <span class="ow">in</span> <span class="n">decoded</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="n">decoded</span><span class="p">)</span>
<span class="k">break</span>
<span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="s">'[^\:\s]\d+.{3}\d+'</span><span class="p">,</span> <span class="n">decoded</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">match</span><span class="p">:</span>
<span class="k">raise</span> <span class="nb">ValueError</span><span class="p">(</span><span class="s">"Invalid expression string"</span><span class="p">)</span>
<span class="n">expression</span> <span class="o">=</span> <span class="n">match</span><span class="p">.</span><span class="n">group</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="c1"># properly handle division
</span> <span class="k">if</span> <span class="s">'/'</span> <span class="ow">in</span> <span class="n">expression</span><span class="p">:</span>
<span class="n">expression</span> <span class="o">=</span> <span class="n">expression</span><span class="p">.</span><span class="n">replace</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="s">'//'</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="nb">eval</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
<span class="c1"># print results to screen to see script progress
</span> <span class="k">print</span><span class="p">(</span><span class="n">expression</span> <span class="o">+</span> <span class="s">' = '</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">result</span><span class="p">))</span>
<span class="c1"># encode and transfer
</span> <span class="n">data</span> <span class="o">=</span> <span class="nb">str</span><span class="p">(</span><span class="n">result</span><span class="p">).</span><span class="n">encode</span><span class="p">(</span><span class="s">'utf-8'</span><span class="p">)</span> <span class="o">+</span> <span class="sa">b</span><span class="s">'</span><span class="se">\n</span><span class="s">'</span>
<span class="k">print</span><span class="p">(</span><span class="s">'Sending: '</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">result</span><span class="p">))</span>
<span class="n">client</span><span class="p">.</span><span class="n">send</span><span class="p">(</span><span class="n">data</span><span class="p">)</span></code></pre></figure>
<h1 id="regular-expression-attempt">Regular Expression (attempt)</h1>
<p>The most important line here was the regex which consisted of the following (see if you can spot the mistake!):</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="s">'[^\:\s]\d+.{3}\d+'</span><span class="p">,</span> <span class="n">decoded</span><span class="p">)</span></code></pre></figure>
<p>This would skip everything proceeding the colon and whitespace and then group both sets of numbers, and the whitespace/operator between them for evaluation.</p>
<p>The response to this was the following:</p>
<p><img src="http://codingo.io/assets/2017-03-08-FastMath-Writeup/FirstRegexResponse.png" alt="FastMath Regex First Attempt" /></p>
<p>I spent far more time on this part of the challenge than I care to admit. I didn’t entirely read my logs and spent my time stuck on the two lines where I have Sending <code class="language-plaintext highlighter-rouge">1766845</code> and the response <code class="language-plaintext highlighter-rouge">459060</code> is <em>not correct</em> and put a lot of focus into trying to identify why a different response was being sent to what was calculated (not the truth, but it’s what I was thinking).</p>
<h1 id="adding-further-regular-expressions-to-catch-xx">Adding further regular expressions to catch x=x</h1>
<p>Eventually I identified that the response to an answer was:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>AnswerProvided is correct
AnswerProvided = AnswerProvided
</code></pre></div></div>
<p>And then the next question would be presented. This was causing issues with my regex as I was passing values from the previous answer into my group, ultimately preventing the correct evaluation from sending.</p>
<p>I resolved this by continuing back to the beginning of my loop if there wasn’t a mathematical operator in my decoded string, putting this after the sentinel check as I didn’t want to miss my flag (which likely wouldn’t contain an operator). This looked like the following:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">if</span> <span class="ow">not</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="s">'[-+/*]'</span><span class="p">,</span> <span class="n">decoded</span><span class="p">):</span>
<span class="k">continue</span></code></pre></figure>
<h1 id="updating-regular-expression-to-handle-more-than-one-digit">Updating regular expression to handle more than one digit</h1>
<p>This got me to question number 10 – at which point I realized that my regex from earlier:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="s">'[^\:\s]\d+.{3}\d+'</span><span class="p">,</span> <span class="n">decoded</span><span class="p">)</span></code></pre></figure>
<p>Would only work if I were to be presented with a single digit question. I updated this to the following:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="p">.</span><span class="n">search</span><span class="p">(</span><span class="s">'[^\:\s]\d+\d+.{3}\d+'</span><span class="p">,</span> <span class="n">decoded</span><span class="p">)</span></code></pre></figure>
<p>Essentially the same as above, but it would now capture two digits over one as I didn’t expect the challenges to go past 99 (although could use \d* if they were to).</p>
<h1 id="flag-reveal">Flag Reveal</h1>
<p>After making the changes above and letting the bot run for fifty questions I was then rewarded with the flag (redacted):</p>
<p><img src="http://codingo.io/assets/2017-03-08-FastMath-Writeup/flag.png" alt="FastMath Flag Reveal" /></p>
<p>Success!</p>
<p>I hope this helped you to better understand how to approach this kind of programming based ctf item. If you’re Brisbane based, or find yourself here be sure to check out <a href="https://www.meetup.com/en-AU/SecTalks-Brisbane/">SecTalks</a>.</p>Category: Coding Points: 300 Solves: 14 Description: crikeyconctf.dook.biz:23776How do you perform arbitrage with Bitcoins?2014-09-28T07:01:58+00:002014-09-28T07:01:58+00:00http://codingo.io/bitcoin/2014/09/28/performing-arbitrage-bitcoin<h1 id="what-is-arbitrage">What is Arbitrage?</h1>
<p>Put simply, arbitrage is buying something at a low price and selling more or less immediately at a higher price through a different market. There are many kinds of arbitrage, but all of them boil more or less to the same goal: Capitalizing on a price difference for economic gain.</p>
<p>Arbitrage is everywhere in our global economy and lots of commercial transactions depend upon it and utilize it. Goods are produced at a reduced cost in one place and are sold at a higher cost someplace else. The important thing is that the differences in price allows us to recoup the costs of the transaction; i.e. packaging, transportation, management, etc. For our discussion we will be referring to the arbitrage between cryptocurrencies between exchanges and other markets.</p>
<h1 id="why-does-it-happen">Why does it happen?</h1>
<p>The reasons for arbitrage are many and opportunities are created every day. Globally, it can be said that these profit windows open due to market inefficiencies and perhaps that’s the best way to put it since the work of many arbitrageurs decreases the profitability of arbitraging, benefiting the market as a whole.</p>
<p>Remember that Arbitrage depends on opportunities and these are not a plentiful resource; it is safe to say that arbitrageurs compete for these opportunities to capitalize on them when possible.</p>
<h1 id="is-it-hard">Is it hard?</h1>
<p>Arbitrage requires both skill and strategic thought. While “simple” arbitrage situations, like buying goods at a farm to sell them at a market may not be the most shining examples, they still require some skill to manage them profitably. Also – remember that an arbitrageur’s worst enemy is another arbitrageur.</p>
<p>This is a game that begins with observation and measurement of the market as the arbitrageur must, at all times, be observant of the price difference in the goods he intends to acquire and sell. The markets move and what was an opportunity before can become quite the opposite in the future; an arbitrageur must hedge the risks he or she takes while doing their business.</p>
<h1 id="arbitrage-in-the-bitcoin-economy">Arbitrage in the Bitcoin Economy</h1>
<p>Certainly there’s room for arbitrage with Bitcoin, but hold your enthusiasm; the golden age of bitcoin arbitrage has ended long ago. In 2014, there are no 30 percent ROI opportunities like in the MtGox days. The essence of current arbitrage remains the same, but the opportunities however are different.</p>
<p>A quick scan of currency arbitrage in the Cryptsy and Poloniex exchanges, as of September 17 2014 at 02:03 GMT yields three potential opportunities. Let’s analyze them.</p>
<p><strong>Opportunity 1</strong>: Buy a Hunter Coin (HUC) at Cryptsy and sell it for BTC at Poloniex. <code class="language-plaintext highlighter-rouge">Profit: 0.00000065 BTC. Margin: 3.96%</code>.</p>
<p>Yes, that’s it – 65 satoshis. That’s 0.0003010475 of a dollar. At this rate you would need 3321 opportunities like this to earn a single dollar.</p>
<p><strong>Opportunity 2</strong>: Buy a ProtoShare (PTS) on Poloniex and sell it for BTC at Cryptsy. <code class="language-plaintext highlighter-rouge">Profit: 0.00000001 BTC</code></p>
<p>In this scenario you would need 215 thousand “opportunities” to make a single dollar.</p>
<p><strong>Opportunity 3</strong>: Buy an Execoin (EXE) on Poloniex and sell it on Cryptsy. <code class="language-plaintext highlighter-rouge">Profit: 0.02293949 BTC. Margin: 11.32%</code>.</p>
<p>This is a fine opportunity indeed; at current exchange rates this is 10.64 USD of money earned in a supposedly risk-free transaction.</p>
<p>However, in the strict academic sense, there’s no arbitrage here. Why? Well, for one, there’s risk. You might not have the coins needed to trade in Poloniex and while you send them over the opportunity might disappear. Every cryptocoin movement between wallets takes time and good opportunities won’t last long. This is the first risk of cryptocurrency arbitrage.</p>
<p>Good arbitrageurs know this and use faster coins to move funds from exchange to exchange. The risk might be less, but it’s a risk nonetheless; while the world of Wall Street’s arbitrage happens in the miliseconds, in the world of cryptocurrencies it happens over minutes of waiting for transaction verification.</p>
<p>Now stop and think about how poor this is. Cryptsy is one of the exchanges with the most trading combinations – otherwise known as pairs – available, it has 298; placing it fifth overall. In addition it handles a very large transaction volume of 1117 BTC daily. This makes it one of the most attractive places to be conducting crypto-arbitrage, as there are hundreds of users trading the numerous alt-coins it supports. Still, there is over half a million dollars in trades going on, but only one true opportunity available – measuring in at just $10</p>
<p>Almost two hours later, at 05:45, the opportunity is still intact. What happened? Why not just go and take it? Now we come to the second risk of bitcoin arbitrage: Exchange withdrawal issues.</p>
<p>This is another layer of danger which inexperienced users usually overlook when thinking about arbitrage. Simply put, the exchanges withhold the right – at least Cryptsy does – of doing whatever they want with their withdrawal services. Let’s travel back in time to March 18 2014, where a young curious would-be arbitrageur scanned an opportunity and was willing to try it.</p>
<h1 id="the-scenario-buy-infinitecoin-at-cryptsy-then-sell-it">The scenario: Buy InfiniteCoin at Cryptsy, then sell it.</h1>
<p>The wannabe did as the program said, and sent his precious bitcoin to the exchange; where he bought the InfiniteCoin he needed, with every fee already taken care of in his calculations. He wrote a destination address, then he clicked on “Withdraw”.</p>
<p>IFC had a block time of 30 seconds – that meant the transaction is supposed to take that amount of time to be confirmed. The amateur arbitrageur waited 30 seconds, and he began thinking that it must be some administrative issue.</p>
<p>60 seconds passed, and he kept thinking the same; two hours passed, and he emailed support.</p>
<p>“Oh, hello Cryptsy? InfiniteCoin withdrawals are taking infinite time. ”</p>
<p>Three hours later they replied saying the withdrawal was at last being processed.</p>
<p>By then though the arbitrage opportunity was lost. However, the story doesn’t end here. Twelve days later, and after several support tickets, the coins were at last transferred. Not only did the opportunity of profit pass, but the coins were more or less worthless by the time they were traded.</p>
<p>Experienced arbitrageurs already know this, in these cases the arbitrage opportunity is inexistent. How do they know? Well, to shed some light on it, sometimes the price is low not because the sellers are mis-pricing it, but because the exchange is broken, and they want their money out.</p>
<h1 id="so-how-do-you-do-bitcoin-arbitrage">So how do you do bitcoin arbitrage?</h1>
<p>There are many ways to do arbitrage with bitcoin, and other crypto-currencies, and they depend on several questions:</p>
<ul>
<li>Will I have to move my currency between exchanges or are they already there?</li>
<li>Will I trade a single pair or several?</li>
<li>Which currency will I end up with? Dollars? Bitcoins? Something else?</li>
</ul>
<p>Let’s take a look at the first question. While it is easy to imagine buying at a discount on one place and selling at a premium on another, it is not as intuitive to think about already having your funds there. This is our first arbitraging method.</p>
<h2 id="method-1-waiting-with-coins">Method 1: Waiting with coins</h2>
<p>This method requires you to have the same amount of the coin you’re buying in the market you’re selling, and performing the trade as simultaneously as possible; back in the example above, it would’ve meant having enough Execoin ready to sell, while having enough Bitcoins to buy it too. Since it is difficult to reliably predict which currencies will have arbitrage opportunities in the often volatile markets, you would need to study the currencies for trends of potential arbitrage.</p>
<p>As it is, this method suffers no transaction risk whatsoever; your coins operate in the exchanges’ systems and are as instant as the exchange allows them to. The biggest drawback this method has is that the volatility risk might eliminate your gains from arbitrage. Requiring 2x the money involved in the transaction is not as big a disadvantage as it may seem, since current arbitrage opportunities are small.</p>
<p>Pros:</p>
<ul>
<li>No transaction risk.</li>
<li>No transaction fees.</li>
</ul>
<p>Cons:</p>
<ul>
<li>The Volatility risk.</li>
<li>Requires much more money on hands.</li>
<li>Idle coins waiting for opportunity earn no return for you.</li>
</ul>
<p>If you are keen on taking transaction risks, you might as well perform the second arbitraging method.</p>
<h2 id="method-2-moving-coins">Method 2: Moving coins</h2>
<p>This is what everybody thinks when they first learn about crypto currency arbitrage. You buy cheaply on Exchange A and then you sell it at a higher price on Exchange B. There’s not much more than that, and that is why the barrier of entry is so low for this method; which means more arbitrageurs, and less opportunities for everyone.</p>
<p>Transaction risk is the biggest here, as explained above there’s no guarantee that certain exchanges will allow you to make use of your funds as readily as you would like to. Also, this does not mitigate the volatility risk you will incur by holding cryptocurrencies. This may change though in the future with cryptocurrencies as they stabilize.</p>
<p>Pros:</p>
<ul>
<li>It’s relatively easy to do.</li>
<li>No transaction fees.</li>
<li>No idle coins waiting for opportunity.</li>
</ul>
<p>Cons:</p>
<ul>
<li>It’s easy enough for everyone to do.</li>
<li>Transaction risk.</li>
<li>Volatility risk.</li>
<li>Transaction fees.</li>
</ul>
<h2 id="method-3-triple-arbitrage">Method 3: Triple Arbitrage</h2>
<p>Sometimes it is not a matter of buying apples at a low price and selling them at a premium elsewhere. Sometimes you buy apples to exchange for oranges and then exchange those oranges for bananas, which you then trade for apples again. Triple arbitrage takes advantage of price discrepancies amongst different trade-able objects. In this case it’s with the many currencies on major exchanges.</p>
<p>It is unlikely (and much riskier) that anyone performs triple arbitrage without automated means, the barrier of entry is also much higher than the two previous methods with a single trading pair.</p>
<p>Pros:</p>
<ul>
<li>You can combine it with the two previous methods.</li>
<li>Less likely for common arbitrageurs to exploit these methods mean more opportunities.</li>
<li>If an exchange is particularly inefficient you might be able to do everything in one place.</li>
</ul>
<p>Cons:</p>
<ul>
<li>Unfeasible without automated means.</li>
<li>Doing this with the “moving coins” method increases transaction risks & fees.</li>
<li>Doing this with the “waiting coins” method requires more capital.</li>
<li>More trading fees.</li>
</ul>
<p>While you might be able to use more than three pairs to trade, the possibilities of doing so diminish due to trading fees. We should carry on to the next question, which mitigates a risk that every method above has.</p>
<h2 id="method-4-staying-in-stable-currency">Method 4: Staying in stable currency</h2>
<p>You might have read the term “going long”, which is trader jargon for staying in a currency or investment no matter the short-term performance. If you believe bitcoin will continue to acquire value in the future, you are thinking in this same mindset – your investment is its own holdings, rather than taking advantage of a market discrepancy.</p>
<p>But in the volatile world of crypto currency arbitrage, you might not necessarily want to go long with Bitcoin depending on your feelings of the market. The volatility for the five year old currency is high and any gains that you acquire from arbitrage are not stable. A mere ten percent gain or loss on the value of Bitcoin could double or destroy your gains from the arbitrage methods listed earlier in this article.</p>
<p>By staying in a stable, government-backed currency such as the US dollar, you can meanwhile get into an exchange, trade for your profit, and get out. Although you’ll be leaving the cryptocurrency market and entering charted, safer waters, there will be much more scrutiny on fees, waiting periods and other financial regulations</p>
<p>Pros:</p>
<ul>
<li>You can incorporate this philosophy into every method.</li>
<li>Much more stable than playing the market.</li>
</ul>
<p>Cons:</p>
<ul>
<li>No opportunity for short selling and gains.</li>
<li>Fees from exchanges to and from fiat currencies</li>
</ul>
<p>There surely are variations and increasingly complex ways of conducting arbitrage, for instance instead of triple-arbitrage, quadruple and quintuple-arbitrage. These ways exposed here, and their resulting combinations, should be enough to describe what’s accessible to most bitcoin traders worldwide.</p>
<h2 id="conclusions">Conclusions</h2>
<p>These methods listed above are, at a glance, how to approach arbitrage with Bitcoin and other cryptocurrencies on the market. In my opinion, the landscape for arbitraging is not easy – Take it from someone who thought it would be profitable and went through the effort of developing software exactly for arbitrage. I could be proven wrong, but I would need to see data to believe it; just as I needed data to conclude the great risk of arbitrage at this time.</p>What is Arbitrage?Proof of Stake – How does it work and what are the advantages?2014-09-19T07:01:58+00:002014-09-19T07:01:58+00:00http://codingo.io/bitcoin/2014/09/19/proof-of-stake-advantages<p>Recently more and more alt coins have decided to integrate a proof of stake system over the traditional proof of work system used by Bitcoin and Litecoin. But why, one may ask? Frankly, the proof of work system is not perfect. A cryptocurrency without proof of stake doesn’t just lack an incentive for users to continue holding on to their coins, but is also susceptible to a 51 percent attack if one party controls a majority of the total mining output. To solve help these problems, some cryptocurrencies, such as NXT, have implemented a 100 percent proof of stake system. Other currencies, such as Dogecoin, have implemented elements of proof of stake to take advantage of its benefits.</p>
<h1 id="what-is-proof-of-stake">What is Proof of Stake?</h1>
<p>Like a proof of work system, proof of stake allows miners to verify block chain transactions and solve puzzles in order to receive rewards – which ultimately are the coins you receive. In a proof of work system, miners complete difficult puzzles using the hashing power of their computer equipment and are rewarded based on how quickly they can solve the mathematical puzzles. The faster the hashrate, the more coins they will receive. Mining in a proof of stake system however is not completely determined by one’s hashrate (or computing power,) but instead by how much of the currency they currently own. If someone owns five percent of the currency, then they can mine five percent of the blocks.</p>
<p>In proof of stake the “mining” that goes on doesn’t necessarily refer to the mining done on currencies such as Bitcoin with powerful computing equipment. Instead, “mining” occurs when transactions take place within the currency, generating fees. These fees are more likely to go to the users with greater stakes in the currency. This creates an incentive for miners to hold onto their coins instead of trading them away as soon as they’re earned.</p>
<h1 id="how-proof-of-take-can-solve-the-flaws-of-proof-of-work">How proof of take can solve the flaws of proof of work</h1>
<p>Although more merchants are accepting Bitcoin, the price has yet to recover from its peak late last year. Though there are many reasons for this, one in particular is because most of these merchants convert their Bitcoins into dollars or other fiat currencies as soon as they earn them. This creates a downward pressure on the price of Bitcoin due to the constant selling of currency that exist the Bitcoin economy. In the proof of work system there is no incentive for merchants to keep Bitcoins and due to its volatile nature many merchants are scared to hold them for longer periods of time. The proof of stake system however gives people an incentive to keep their coins rather than selling right away.</p>
<p>Proof of stake can also solve the very critical risk of 51 percent attacks that haunt both the Bitcoin and other cryptocurrency community. Earlier this summer the Ghash.io pool, controlled by CEX.io, came dangerously close to reaching the 51 percent attack potential. While the company did eventually reduce its network hashing rate and issued a press release noting its intentions to prevent a 51 percent attack, proof of stake would permanently prevent this issue from ever occurring. This is because it is not only very difficult to attain, but would do more harm to the causer of the 51 percent attack than to the rest of the network.</p>
<h1 id="the-advantages-of-a-mix-of-both-systems">The advantages of a mix of both systems</h1>
<p>The proof of stake system is by no means perfect. By giving people an incentive to save their coins it also gives them an incentive to hoard and never spend any coins. This in turn lowers the overall transaction volume, which can hurt the price of the currency as well. Ultimately there must be a healthy median between these two systems that minimizes the flaws of both and maximizes their advantages. But will a currency like Bitcoin ever adopt a proof of stake system? Probably not. This is because changing anything in the Bitcoin protocol as dramatic as a new proof of stake system would make the currency seem weak and unguided. But it is very likely in the future that other currencies will adopt a mixed system between the two methods, much like Dogecoin.</p>Recently more and more alt coins have decided to integrate a proof of stake system over the traditional proof of work system used by Bitcoin and Litecoin. But why, one may ask? Frankly, the proof of work system is not perfect. A cryptocurrency without proof of stake doesn’t just lack an incentive for users to continue holding on to their coins, but is also susceptible to a 51 percent attack if one party controls a majority of the total mining output. To solve help these problems, some cryptocurrencies, such as NXT, have implemented a 100 percent proof of stake system. Other currencies, such as Dogecoin, have implemented elements of proof of stake to take advantage of its benefits.An error occurred. Detailed message: 1 conflict prevents checkout when doing a GIT pull request in Visual Studio2014-05-27T07:01:58+00:002014-05-27T07:01:58+00:00http://codingo.io/git/visual-studio/2014/05/27/git-conflict-prevents-checkout<p>I’ve noticed something interesting happening in Visual Studio 15 when I move between environments and try to do a pull request.</p>
<p>I’m able to see the incoming commits queue as well as the content within them but when I go to do a pull request I’ll receive the error:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>An error occurred. Detailed message: 1 conflict prevents checkout
</code></pre></div></div>
<p>This problem occurs when you try to checkout a file on a branch in your local repository but is out of sync with the same branch on the remote repository (sidenote: stop working directly in master and this issue is nearly always gone!).</p>
<p>Although this is usually quite an easy problem to resolve for some reason Visual Studio won’t show the diff tool in this scenario, nor will it allow a pull request within the GUI to update a local repository.</p>
<p>Thankfully as we’re using git for our version control backend this is quite easy to resolve in the backend.</p>
<p>To overcome this first open a new command line window for the repository where the conflict is.</p>
<p>To do this navigate to the synchronization page which shows you incoming and outgoing commits. In the window that appears click <em>Actions</em> and then <em>Open in Command Prompt.</em></p>
<p><img src="http://codingo.io/assets/2014-05-27-Conflict-Prevents-Checkout/Git-Sync-Window.png" alt="Git pull conflict" /></p>
<p>You then want to perform a <code class="language-plaintext highlighter-rouge">git status</code> command to confirm that you have outstanding changes to pull from the remote branch.</p>
<p><img src="http://codingo.io/assets/2014-05-27-Conflict-Prevents-Checkout/Git-Status.png" alt="Git Status" /></p>
<p>Then do a <code class="language-plaintext highlighter-rouge">git pull</code> to manually pulldown the latest commits to the branch to your local repository. In ordier to do successfully perform a pull request you will also need to enter alternative access credentials. If you don’t already have these (or know them) then logging into your Visual Studio Online page, managing your profile (top right) and setting them on Security will create the credentials you need whenever using git from the console window.</p>I’ve noticed something interesting happening in Visual Studio 15 when I move between environments and try to do a pull request.Bitcoin Wallets: What are the differences between them?2013-01-02T07:01:58+00:002013-01-02T07:01:58+00:00http://codingo.io/bitcoin/2013/01/02/bitcoin-wallet-differences<p>Satoshi Nakamoto first published their paper on Bitcoin back in 2009, but it didn’t reach real mainstream popularity until recently. Only the users who understand the highly technical mechanics of Bitcoin invested their time in the technology during 2009-13 intermission period. In February of 2013 Bitcoin, and soon cryptocurrencies in general, finally entered the public consciousness. Despite being a four year old technology, many of the core features of the Bitcoin protocol were not polished in any capacity, let alone ready for mass public consumption. The Bitcoin wallet is an excellent example of one such component. Since that first popularity explosion, much expertise has gone into designing and refining the Bitcoin wallet. Today we will be taking a look at the differences in functionality of the three types of wallets as well as comparing their security and usefulness. These types of wallets suit users depending on the way people wish to access their funds, as well as who they want to entrust the long-term security of their accounts.</p>
<h1 id="what-is-a-bitcoin-wallet">What is a Bitcoin wallet?</h1>
<p>For a majority of Bitcoin users, the first cryptocoin interface they encounter is the wallet. Much like the physical wallet you keep in your back pocket, your Bitcoin wallet serves to store currency that you mine, purchase, or receive from others. Your digital wallet functions in a very similar way to its real life counterpart – however,there is one key difference: Cryptocurrencies are exchanged using addresses not unlike email, with the exception that your Bitcoin addresses are generated randomly on your behalf. Your wallet can hold more Bitcoin addresses then you could ever use, and each address functions as a separate place for you to send and receive money. Think of each address as a separate compartment in your wallet, or as the various different bank cards you own that each have their own individual balances. These addresses are where all transactions are directed, whether they’re yours, or run by the companies you use to collectively hold your money.</p>
<h1 id="offline-wallets">Offline Wallets</h1>
<p>In the beginning of Bitcoin, there were offline wallets. These wallets operate in the “traditional” wallet style – that is, with all of your wallet data stored only locally on your own personal machine and not in need of a constant internet connection. You do need to be on the internet to receive and send your currency, but you don’t always have to be online. Initial Bitcoin commerce wasn’t possible without at least a basic wallet, and the offline wallet was the simplest solution for early adopters to code up and release to the public, including the one Nakamoto published. As such, offline wallet apps are readily available on both the desktop and mobile platforms. It should also be noted that any damage to your wallet data can quickly render it corrupted, which means your keys (and your cash) are unrecoverable without a backup. This limitation drives fear into the heart of many Bitcoin enthusiasts, some of which can barely keep track of their physical keys, let alone their digital ones. As a result, it’s important to keep many copies of your wallet information secure and safely stored away, even if just on a USB drive behind a safe.</p>
<h1 id="online-wallets">Online Wallets</h1>
<p>When hosted, or online, wallets entered the scene, they changed the entire Bitcoin wallet paradigm. Instead of keeping all your wallet information stored locally, a sensible online wallet stores an encrypted copy of your wallet data on their servers with a password you choose. When you want to access your wallet, your data is downloaded and decrypted locally. After you have completed your transactions, the hosted wallet uploads your encrypted wallet to back servers the hosted wallet controls. This relieves the user of the responsibility of backing up their wallet as well as “syncing” their wallet across their devices — all while keeping the users keys protected from abuse on the company’s servers by only doing encryption and decryption operations on the “client side”, aka locally on your machine. Online wallets are most popular on web browser centric and mobile devices, but the nature of this type of wallet makes easy access possible on any networked system. IT also ensure you don’t have to run your computer all day as a node to keep up with transaction activity on the Bitcoin network, and can let your provider handle all the trouble. However, if anything were to happen to the company you use to store your currency, then you’d be out of luck unless they offer a backup service to store copies of your wallet offline.</p>
<h1 id="deterministic-wallets">Deterministic Wallets</h1>
<p>A third type of wallet exists exclusive of the other two: The deterministic wallet. The previous two types of wallet operate as variations of each other, where the online and offline wallets manage the same wallet data that is located either on your machine or is shifted to someone else’s. On the other hand, the deterministic wallet generates master root key (consumable by humans in the form of a twelve word passphrase) and hands it to the user. The root key is the only information the user is required to keep track of. Once the root key in input, the deterministic wallet uses advanced cryptographic algorithms to derive new private keys, or addresses, from the original root key. Because the wallet is creating new addresses using the same formula with the same initial conditions each time, all addresses are said to be predetermined from the root key. This handy fact allows you the option to store your wallet backup securely in your brain (via memorization) without the need for the key to exist digitally anywhere — all while still being able to access funds in any of the derived addresses. The portability of this system grants deterministic wallets secure interfaces on your desktop, your mobile phone, and your web browser. This of course still depends on you remembering a very unique kind of password that may be difficult to recall, or if written down can pose a security threat.</p>
<h1 id="which-should-you-choose">Which should you choose?</h1>
<p>Since the invention of the Bitcoin protocol users and developers alike have worked to shape the function and feel of cryptocoin wallets. Today wallets are more robust, packed full of features, and offer security that we couldn’t imagine during the infancy of Bitcoin. Wallets come in many forms and serve many purposes. We’ll share with you next how most popular wallets available compare, and what you can do to pick the one that suits you.</p>Satoshi Nakamoto first published their paper on Bitcoin back in 2009, but it didn’t reach real mainstream popularity until recently. Only the users who understand the highly technical mechanics of Bitcoin invested their time in the technology during 2009-13 intermission period. In February of 2013 Bitcoin, and soon cryptocurrencies in general, finally entered the public consciousness. Despite being a four year old technology, many of the core features of the Bitcoin protocol were not polished in any capacity, let alone ready for mass public consumption. The Bitcoin wallet is an excellent example of one such component. Since that first popularity explosion, much expertise has gone into designing and refining the Bitcoin wallet. Today we will be taking a look at the differences in functionality of the three types of wallets as well as comparing their security and usefulness. These types of wallets suit users depending on the way people wish to access their funds, as well as who they want to entrust the long-term security of their accounts.