GitHub Accidentally Exposes Some Plaintext Passwords In Its Internal Logs - Slashdot

Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

GitHub Accidentally Exposes Some Plaintext Passwords In Its Internal Logs (zdnet.com) 149

GitHub has sent an email to some of its 27 million users alerting them of a bug that

. "During the course of regular auditing, GitHub discovered that a recently introduced bug exposed a small number of users' passwords to our internal logging system," said the email. "We have corrected this, but you'll need to reset your password to regain access to your account." ZDNet reports:

You may like to read:

• How can a clear text password be available to them at all to record it in a log?

  • • Are you sure it's correct? How would you log in users? They would hash their password on the client side and send the hash so that the back end would compare the hashes?

      • by Anonymous Coward writes:

    • What should happen is that a hash gets calculated on client AND server side, you don't want to store the client's hash directly in the database... for obvious reasons.

      • by Anonymous Coward writes:

        What? No. The server holds the hashed user's passwords so anytime a user types his password it compares its hash with the one it got and only then it allows the login.

        • Single hashing bears a problem. If done on the server side, the clear password has to travel to the server. In an encrypted tunnel, all right, but you have to trust that tunnel and you have to trust the process between tunnel and hashing, which as we can see here has been compromised.

          Hashing on the client side reduces the problem to one akin to storing a plain-text password where you compare what is sent by the client with the database, leaving the data vulnerable in case of a database hack because knowing

          • Double hashing does effectively nothing useful. The correct solution if you don't trust that the tunnel is encrypted is to use a zero knowledge password exchange protocol like SRP.

            • Problem with SRP is that it's not particularly useful in a browser, practically speaking.

              The implementation actually performing the SRP is going to be a javascript download from the site you are dealing with. Meaning the server and connection you explicitly do not trust to send your password to is the one you are trusting to provide the alternative implementation.

              Doing SRP right requires a lot of work that just isn't happening. Namely browsers particularly need to implement some sort of industry standard

              • I keep seeing responses like this but they never make sense to me.

                Your argument basically boils down to "JS-based SRP is insecure because JavaScript crypto is insecure, which is because of the web page delivery mechanism". Well, yes, web crypto has a chicken and egg trust issue, but this problem *already exists in the current traditional login methods*! If a malicious actor can inject HTML/JS into your login page, *both* SRP and traditional password logins would both be hosed, since it would be trivial to j

        • That's either bad security or you're vastly oversimplifying. Assume everyone is listening in on the network transaction; so you can't send the password over the network, and you can't send the hash over the network, otherwise the transaction can be replayed to spoof the server. Each session should be unique.
        • • This entire thread is a fucking trainwreck but your comment takes the cake. Hashing with a key?! Hashing locally and sending the hash to the server so that it compares hashes? Then your hash is the damn password and you're back at sending your password to the server, who stores it in plaintext. Yay.

    • If you hash on the client, then you're not really hashing, you're just using a different password and the thing in your database is now the real password, because that's what the client presents to log in. If you really want to do it properly, there are zero-knowledge protocols that allow it, but simply hashing on the client is not sufficient.

      • I always wondered why we don't hash client-side *and* server side. Then the password would be the result of a hash, as you say, and it would also be stored properly.

        Being hashed client-side also means the server doesn't have to worry about too-long passwords or bad characters in the password or anything; it just expects the result of a hash. As long as the thing it receives is (say) 256 bits long, it's good.

        • There are a couple of reasons, but the main one is that if you're able to run code on both the client and server then there are much smarter ways of doing this. The simplest way is to store a public key on the server and store the corresponding private key on the client. Then the server sends some data to the client, which encrypts it with the private key and sends it to the server, which decrypts it with the public key. If the encrypted data matches the data provided by the server, this proves that the

        • I always wondered why we don't hash client-side *and* server side. Then the password would be the result of a hash, as you say, and it would also be stored properly.

          Because it would only be stronger than really weak passwords. While the result of a SHA1 hash is stronger than "password", knowing the exact length of the password greatly reduces the number of possible passwords, therefore making it easier to brute force.
        • • • • Probably. There are 11 possibilities, just within the realm of computing, and that's if we limit to only those listed on that wiki page, though; so it's a bit irresponsible to simply use the acronym in this case.

                • You poor thing. Let me dissect those 11 possibilities for you, in an attempt to find the one that matches the context of password authentication:

                  Single responsibility principle, a concept in computer science

                  Hmm. Could that be it? Does "single responsibility principle" sound related to "password authentication"? Hard to tell, but I think it doesn't.

                  Secure Remote Password protocol, a protocol for authentication

                  Hmm. Could that be it? Does "secure remote password, a protocol for authentication" sound related to "password authentication"? No way.

                  • and that's if we limit to only those listed on that wiki page

                    In other words, there's no guarantee it's any of those 11.

                    That said, I really didn't need the breakdown, but many others here may have, so thank you for providing it, I guess?

                    Asshole.

                  • So you proactively complained, on behalf of others. That's actually worse.

                    but many others here may have

                    Yeah, all the lesser minds, common idiots who -- unlike you -- lack the intelligence to resolve an acronym with the help of a disambiguation page.

                    You sound and act incredibly arrogant.

                  • So you proactively complained, on behalf of others. That's actually worse.

                    Howso?

                    You sound and act incredibly arrogant.

                    Hi, Pot, I'm Kettle!

      • This is why you don't have novices implement security. Also make a note that an expert on the use of an SSL API is not the same as a security expert. We continue to have these security problems because there are so many people implementing security who aren't security experts. Never mind the actual design itself which should come from even higher up.

    • If you have users who turn off JavaScript-by-default for security reasons or software freedom reasons, how do you hash on the client side?

      • <noscript>This website requires Javascript enabled in your browser, you fucking twat. Please enable it and try again. If you don't wanna enable Javascript you can go and fuck yourself with a cactus.</noscript>

        Problem solved.

        • This is github, so when I'm using HTTP auth via git, does git need to implement a javascript engine?

        • "JavaScript is enabled, but I use LibreJS [gnu.org] to block proprietary script [gnu.org]. Can you send me a copy of the authentication script's unobfuscated, unminified source code under a free software license so that I (or another member) can perform a code review before executing it?"

          If the answer is no, then this is why GitHub has been rated F for requiring execution of proprietary script [gnu.org].

          • • • and never accept free beer.

                I have to intervene at this point, what you call "free beer" should in fact be called "gratis beer".

                Thanks for caring.
    • • • The only way, to end up with the same value, is to enter the same inputs

          Or one of the infinite number of other possible inputs that will produce the same hash. But please go on educating us on this matter. I mean how hard can it be, right?

  • I guess http basic auth over TLS.
    The connection is encrypted using TLS but the password is transferred in the clear (base64).

    Don't use basic-auth.
    If you absolutely have to, use an application specific password with restricted rights.

    • Basic auth is an HTTP header, and HTTP headers are just as protected by TLS as response headers and bodies. Otherwise, HTTPS would be ineffective against Firesheep-style attacks that clone a session cookie. The other common means of authentication is submitting a password that has been entered into a field of an HTML form as part of an HTTP POST request body. What's any more "in the clear" with HTTP basic authentication than with the form route?

      And in case you believe both forms and basic authentication oug

  • by Anonymous Coward writes:

    How can a clear text password be available to them at all to record it in a log?

    Quite easily.

    Change your form to send a GET request instead of a POST, and *blam* form contents go into the server log as part of the request URI.

    With a normal form submit, this would, of course, be noticed immediately by the user (and one would hope by the developer and QA too) because the form contents show up in the browser URL bar, but if the request is made via Ajax, then this doesn't happen so it could get missed.

    Another way it can happen is a debugging statement left in the code that dumps the user o

  • How can a clear text password be available to them at all to record it in a log?

    Er, because the user clicks "submit" on a password change form? And somebody was debugging something, and was logging the submitted data before the password got hashed?

  • Every single web site you use including Slashdot, Gmail, etc etc has the plain text password available to the server. The hashing of the password with a salt runs on the server. This Github bug was simply that a developer wrote the plain text password to a log file during the authentication process. They didnâ(TM)t store the plain text password in a database.

    • It makes me nervous enough that I think basic hashing and encryption should be done at the client side as well. This would stop everything that isn't tampering with the site itself.

      • No, it wouldn't.

        Let's say your password is X, and you hash it locally to Y. You still have to send Y to the server to be validated. There's no way around this.

        If you are sending Y, then it is possible that the server could potentially log Y, and someone who sees those logs might in turn be able reproduce your login by sending Y themselves, even without ever knowing what X is.

        That Y might happen to be a binary string that has no visible representation or ability to directly type on a keyboard has n

        • Let's say your password is X, and you hash it locally to Y. You still have to send Y to the server to be validated. There's no way around this.

          You don't ever need to send the password, X or Y, if you can depend on running some JS on the client: just use a zero-knowledge proof of identity. However, it's pointless overkill, as man-in-the-browser attacks are more common than MITM attacks these days. Complicated login approaches don't really help against the common attack, so there's little point in getting clever - just make sure you're using HTTPS and POST not GET.

          • Anything that you send to the server can be read by that server and can therefore be potentially logged as plain text (which is what happened here). The only way around this is to move 100% of the server functionality to the client, which means that the client can't exchange any data with the server in any way (because any data that they do exchange to somehow confirm identity can be logged, as I said above).

            • Yes, yes, which is why a method of authentication that is not replay-able is preferred, all things being equal. You're missing the point of "zero-information" protocols: log whatever you want, or send it in the clear, it gives an outsider zero information that's of any future use.

              • It doesn't matter if you send it in the clear or if you send it encrypted... the point is that the server will know what you sent, and if that server logs what you sent, then all someone else will have to do is send the same thing in their own session. If you can't replicate it, then there's no way for the server to possibly be able to verify your identity at all.

                • Protip: when you don't understand the words being used in the post you're responding to, you might develop the habit of looking them up, [wikipedia.org] rather than continuing to demonstrate your ignorance.

                  FFS, at least understand what a replay attack is, and what a non-replay-able protocol is, before wittering on about them.

                  • Second protip, don't move the goalposts of a discussion and then try and call someone on not knowing what they are talking about.

                    The suggestion above was to encrypt the password on the client side.... but of course, you still need to actually send that encrypted password, and that encrypted password could be logged. The server would never know the "real" underlying password, but a person who viewed that log could send that exact same byte sequence without ever ever knowing the alleged real password, ju

                  • C'mon, you said "there's no way around this", and I just pointed out "yes, there is a way around this", where it would be safe to log whatever. And if you were using JS to hash client-side, you could just do it the right way instead.

                    If you really want to solve this kind of problem with client-side JS, you can indeed solve it that way, and make it safe to log everything sent. I think that's solving the wrong problem, but you could.

        • The point isn't to prevent the server/company from getting your hash, but to make that hash useless to other third parties if it gets leaked. If you sent a hash to GitHub and GitHub inadvertently logged it, well, they had your hash to begin with so it's not any extra information, and if it gets leaked, well, at least the third parties don't have your password.

          Wouldn't this work?

          Server stores a salted hash based on username + password.

          Client sends a salted hash based on username + password.

          Server checks salt

          • My point is that if you are sending the server the hash then effectively, that hash *IS* your password... and that's all that a would be attacker would ever have to send to impersonate you, because that's all that *YOU* sent to identify yourself.
  • • by Anonymous Coward writes:

  • How do you think they both check your password and allow you to change your password without ever having the clear text password available?

    It's a trivial thing to do, by not thinking what you are doing through. For example, "Hey Bob, can you work on bug 127654 the "password reset fails in this obscure case" one?". Bob then adds "log(form.getAll().toPrettyString())" to the reset code and either leaves in his debugging output when he checks in the actual fix or modifies production directory if it's a cowboy d

    • By sending them a hash of the password I enter, which is then stored as a hash of that hash in their database. The idea is that even if I am the average dumb user that reuses the same password over and over, the salted hash ensures that this password is not published even if the first hash gets published (or logged, as in this case), while the second hashing ensures that knowing that first hash does not immediately compromise the authentication process.

      • So both requiring running javascript on the browser to login and not actually providing any additional security for them (the hash is now the clear text password).

        So to protect people who reuse passwords we prevent people who don't want to run javascript from logging in at all? Those people being protected could just use their own software to get the same effect without stopping non-js users from accessing the site at all.

        • You could simply allow people not wanting to run Javascript to transmit their plaintext passwords.

          Though that also eliminates any and all grounds for complaining if their true and tried and often reused "password1234!" appears in a log...

      • A string of 32 random hexadecimal digits is stronger than "password123", but it's weaker than a strong password.

  • Generally speaking, if a scheme means the server is never handed the password at all, then a compromise of their database leaks a viable set of credentials to use to log into their service. The alternative to one-way hashing, with password provided to validate knowledge of the password is generally some variation of shared secret, where the server must have the same knowledge the client has. It may be a secret derived from the password and thus the password is still not known, but it's of small comfort wh

  • It sounds like the logs in question were of the form fields being submitted by the user password reset form. Even if you're using SSL (which they are) and the passwords are only stored as a hash (which it sounds like is true at GitHub), there's still a moment where the server has the plaintext password. Some logging frameworks will automatically omit or redact the value of any field like `secret` or `password`. Perhaps in this case they forgot to also redact `password_confirmation` or something.

  • I worry about this, but I see it far too often. We learned decades ago with Unix to never store the plaintext password anywhere. Instead you store a value derived in some from the password and other information (such as a secure hash function). That way even if you get a glance at the password file you never see the passwords. It's basic freshman level security.

    That a web site even has a plain text password is bad, that it stores the password is even more bizarre.

    Sadly, I think there are major tools that

  • Seen this happen a zillion times... person enters password in the "Username: " field.
    Eg (synthesised, from memory):

    Failed login for "secret" on tty04
    Successful login by "jbloggs" on tty04

    • Ok, that's a different beast, that's user error.
  • • A better question is why doesn't the HTML standard for password fields allow automatic hashing with a custom salt?

      It does; it's called digest authentication [wikipedia.org]. But depending on how digest authentication is implemented, it is vulnerable to one of two attacks. If the realm portion is fixed, digest is vulnerable to a replay attack that passes the hash [wikipedia.org]. If the realm portion is variable, it requires the server to store the unhashed password. In addition, digest authentication still uses MD5, which is deprecated and whose immediate successor (SHA-1) is also deprecated.

• the bug "resulted in our secure internal logs recording plaintext user passwords when users initiated a password reset."

  ...

  "We have corrected this, but you'll need to reset your password to regain access to your account."

  Er... are you really sure that this has been corrected?

  • • While I can't prove that your paranoid speculations are incorrect, neither can you prove that they are correct. I believe that you are honestly reporting the way you perceive your experiences, but to claim that this is happening at a site that you don't know personally is, as I said, paranoid. And, frankly, usually personal issues are more important than technical competence at getting promoted, so if that's important to you, you might want to work on personality. I don't think "just slap the word 'secur

    • PIN numbers

      Sorry, lost all credibility right there.
• • • So how is this "random salt" recovered when you need to check the password's validity?

      It's stored along in the data base.
      Most stored password have a form like :

      ${type of algorithm used}${parameters used}${data}

      where:
      - "type of the algorithm used" tell you what was used to generate this (e.g: using Bcrypt, like GitHub as mentioned in the summary).
      - "data" is the actual salted-output that you need to replicate to successfully log-in
      - "parameters" is any extra-data that the algorithm needs to generate password checks.
      Like the salt.
      Or like the number of iterations. Because nobody sane actually use a hash function such as SHA512 anymore. Instead you use a Key Derivation Function (KDF) such as Bcrypt (or Scrypt or Argon2) and those are *slow* on purpose, to make brute-forcing much less likely (e.g.: they slow down by repeating a hash for large number of iterations).

      The exact implementation vary (the above is typically used by the "crypt" function used, e.g., on Linux log-ins),
      but basically are the same : the salt (and iterations) are stored together with the "hash" that you need to test.

      And most of the KDF function can work as "hash_to_compare = KDF(password_login_attempt, old_hash_from_database)", ie.: they can automatically extract the parameters if you give them the string that is in the database, and generate the hash the exact same way.
      They'll invent a new salt (and guess the optimal number of iterations) only if you omit the old hash and give the new password as the single parameter.

    • The salt isn't secret, it's just used to prevent rainbow tables from being useful. If you store passwords as unsalted hashes, then an attacker can construct a large table of all of the hashes of 8-character inputs and compare each of your hashes against their table. If there's a match, then they have a password that will work. If you add a salt, then they can't use such a table, because they have to check each 8-character sequence with the hash prepended. If the salt is different for each password (as it should be), then there's no benefit from pre-calculating the table. If it takes 2 hours of GPU time to compute the table, then with unsalted passwords that's a one-time cost and you can then crack any weak password in a leaked password database almost instantly. In contrast, it will take you 2 hours to attempt each password and crack each weak one in a leaked salted password database.

    • So how is this "random salt" recovered when you need to check the password's validity?

      In addition to what others have said about each password having a random salt stored alongside, you can combine the salt with another string that's stored in a secure hardware module. This string is the same for all passwords, but it can only be accessed by the application. This makes determining passwords even more difficult for someone who gets a copy of the database, since they don't have the entire string that was passed to the hashing algorithm.

      Because computer scientists think they're funny, they c

  • you feed the string and the salt into an encryption algorythm like sha512 which produces a HASH this is what gets stored

    Argh!
    No!
    NO!!!
    NO-NO-NO-NO!!!!
    DO NOT USE HASHES ! (like Sha512).

    These are designed to be *fast* (1), meaning that it could be not impossible for an attacker to guess the password out of the hash simply by brute forcing all the most common password and variations thereof into the same salt and see if they match.

    (1 - And remember that the "tera hash" that ASIC bitcoinminer are reporting are exactly that : trillion of SHA256-like computation per second.)

    USE KEY-DERIVATION FUNCTIONS (KDF) INSTEAD !

    Like the Bcrypt use by github as mentionned in the summary. Or Scrypt (same used by tarsnap). Or Argon2. etc.
    These also produce a value out of a password and a salt, but they are on purpose extremely slow (E.g.: by repeating a hash function over and over for a high number of iteration).
    If each computation takes some time, it doesn't impact login that much (After all, you only need to log in once at the beginning of your session), but it hinders anyone wanting to brute force your password out of a stolen hash.

    It makes data breaches that managed to steal your user database a lot less dangerous (because once you have successfully guessed the password from the hash, the next step is to see all the other places where the user has re-used the same password).

    • USE KEY-DERIVATION FUNCTIONS (KDF) INSTEAD !

      Like the Bcrypt use by github as mentionned in the summary. Or Scrypt (same used by tarsnap). Or Argon2. etc.

      Is SHA256 inside PBKDF2 [wikipedia.org] still safe?

      • PBKDF2 uses SHA-variants in it iteration.
        Despite "Shattered", it's not "broken" yet.

        There are just better more modern KDFs (like the Bcrypt used by Github, like the Scrypt designed for use in tarsnap, or like Argon2 which is the latest competition winner) that don't have PBKDF's short comings (e.g.: collision of long input pass phrases and their SHA-1).

        Regarding : "Shattered" you have to understand its context.
        SHA-1 has known to be not as secure as it could be (a 128bit SHA-1 has not 128bits of security) fo

        • Is Argon2 considered ready for prime time yet, or is it still in the early adopter phase?

    • Don't tell people not to use hashes. The next think they'll think is "Oh, I should use plaintext instead".

      A key-derivation function is also insufficient, since the output is only as strong as the input. Meaning if you have a 10-bit password, the resulting KDF strength will still only be 10 bits.

      You must use a Password-Based Key Derivation Function. A PBKDF can add ~10 bits of security to a password. So if a user gives "password" as their password (2 bits), the resulting hash has ~12 bits of security.
    • • I don't really see why people are so against to hashes that they need to shout.

        My main reason was for commically over-exagerated "hysteria".

        The actual reason why people are against hashes, is a combination of three factors :

        1. Brute forcing:
           • The computational power has vastly increased recently. Noticed the "Tera hash" that bitcoin ASIC miner have been advertising ? That's trillion of SHA256 hashes computed by second. GPU and FPGA are widely available and are good at brute forcing millions, perhaps billions of hashs per second.
           • We humans suck at picking passwords. You don't need to actu

  • 3) you feed the string and the salt into an encryption algorythm like sha512 which produces a HASH this is what gets stored

    Except sha512 is a hashing algorythm not an ecryption algorythm.

  • IIUC, what happened is that the passwords that were revealed were of those who were resetting their account passwords. So this would mean that they needed to be send the new password in clear text to the site, at which point it was logged before being hashed.

    This is still a problem, but I'm not sure that there is any reasonable way around it. It just shouldn't have been logged.

• OK, that's naughty and needs fixing, but it's internal logs, did it need a slashdot story?

  • • Fortunately, investors don't care about the simple stuff.

      • • Exploits are even simpler?

        • Because investors decide what gets done (people need to be paid for the work they do) and they don't care about security.

• GitHub has sent an email to some of its 27 million users alerting them of a bug that exposed some user passwords in plaintext. "During the course of regular auditing, GitHub discovered that a recently introduced bug exposed a small number of users' passwords to our internal logging system

  Is 27 million a small number?..

  • GitHub has sent an email to some of its 27 million users

    Emphasis mine.

• Logging a password is a beginner's mistake, like SQL injection. I found the same bug in unreleased code many years back, and raised it to management so we could track down the engineer who did it. It's the kind of (cough) mistake that can be the "straw that broke the camel's back" when dealing with an engineer who has (cough) "negative productivity."

  Ideally, this kind of bug should be caught in code reviews. As someone who reviews a lot of code, even I'll admit that it's possible for something like this to

  • • • 2-factor authentication can be very useful for improving security. It's burdensome to set up, and makes automation much more difficult. Github, gitlab, and Sourceforge, and other public source control repositories also have a difficult situation _supporting_ multiple access methods,. Even as they improve the security of one system, they're still left with all the potential holes of the other supported systems. From the source articles, Github developers should have caught this one. But let's be cautious to

• I got the email.

  I was impressed that it was handled quickly.

  I'm even more confident because I actually use a proper password manager making sure I have unique passwd's for everything.

• I give them credit -- here they found their own security issue before it became a breach, they fixed it, and they didn't sweep it under the rug but instead they notified their users. Kudos for being forthcoming.

• "It's unclear exactly how this bug occurred"... riiiiiiiight, git blame?

I bet the human brain is a kludge. -- Marvin Minsky

Working...