Server compromised, expect downtime - Discussion
Server compromised, expect downtime
Daniel M Ingram, módosítva 3 év-val korábban at 2020.11.24. 0:05
Created 3 év ago at 2020.11.24. 0:05
Server compromised, expect downtime
Bejegyzések: 3278 Csatlakozás dátuma: 2009.04.20. Legújabb bejegyzések
Dear DhO,
Our server on Hetzner is compromised, they tell me, so will shut this server down shortly until we can get this sorted, I think.
Apologies for the interruption.
Best wishes,
Daniel
Our server on Hetzner is compromised, they tell me, so will shut this server down shortly until we can get this sorted, I think.
Apologies for the interruption.
Best wishes,
Daniel
Chris M, módosítva 3 év-val korábban at 2020.11.24. 7:17
Created 3 év ago at 2020.11.24. 7:17
RE: Server compromised, expect downtime
Bejegyzések: 5406 Csatlakozás dátuma: 2013.01.26. Legújabb bejegyzésekDaniel M Ingram, módosítva 3 év-val korábban at 2020.11.24. 10:37
Created 3 év ago at 2020.11.24. 10:37
RE: Server compromised, expect downtime
Bejegyzések: 3278 Csatlakozás dátuma: 2009.04.20. Legújabb bejegyzések
Ok, the warnings went away when a WordPress site that is also hosted there was updated, so may have been that simple. Will hopefully know more today and will check around to make sure all is well. Let's presume all is well for the moment, but will keep you all updated. If the DhO does need to go down for maintenance and the like, I will post something on my Twitter account @danielmingram about what is going on and when it might be back up.
Chris M, módosítva 3 év-val korábban at 2020.11.24. 11:10
Created 3 év ago at 2020.11.24. 11:10
RE: Server compromised, expect downtime
Bejegyzések: 5406 Csatlakozás dátuma: 2013.01.26. Legújabb bejegyzésekDaniel M Ingram, módosítva 3 év-val korábban at 2020.11.24. 11:11
Created 3 év ago at 2020.11.24. 11:11
RE: Server compromised, expect downtime
Bejegyzések: 3278 Csatlakozás dátuma: 2009.04.20. Legújabb bejegyzések
Ok, was wrong, not fixed, heck. Got a message last night saying was ok, now a new message that it is not.
Hopefully, Simon will be able to sort it.
Thanks for your patience!
Hopefully, Simon will be able to sort it.
Thanks for your patience!
Daniel M Ingram, módosítva 3 év-val korábban at 2020.11.24. 11:48
Created 3 év ago at 2020.11.24. 11:48
RE: Server compromised, expect downtime
Bejegyzések: 3278 Csatlakozás dátuma: 2009.04.20. Legújabb bejegyzések
Ok, after Simon was kind enough to do some excellent tech support, it appears that the problem is Liferay, so it will need to be upgraded to Liferay 7, and the last time we tried this we failed completely, so I will engage some paid expertise to sort this out. If people wonder what I do with the little bit of money I make on MCTB2, it is this sort of thing.
Thanks for your patience.
Expect DhO instability in the coming days. Save large new posts that you care about in a text file or something like that.
We have backups through today, so that's good.
Best wishes,
Daniel
Thanks for your patience.
Expect DhO instability in the coming days. Save large new posts that you care about in a text file or something like that.
We have backups through today, so that's good.
Best wishes,
Daniel
Angel Roberto Puente, módosítva 3 év-val korábban at 2020.11.24. 12:04
Created 3 év ago at 2020.11.24. 12:04
RE: Server compromised, expect downtime
Bejegyzések: 281 Csatlakozás dátuma: 2019.05.05. Legújabb bejegyzésekPapa Che Dusko, módosítva 3 év-val korábban at 2020.11.24. 12:13
Created 3 év ago at 2020.11.24. 12:13
RE: Server compromised, expect downtime
Bejegyzések: 3048 Csatlakozás dátuma: 2020.03.01. Legújabb bejegyzésekAngel Roberto Puente:
Chris Marti:
Were any of our credentials compromised?
LOL
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/smile.gif)
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/smile.gif)
Linda ”Polly Ester” Ö, módosítva 3 év-val korábban at 2020.11.24. 12:33
Created 3 év ago at 2020.11.24. 12:33
RE: Server compromised, expect downtime
Bejegyzések: 7134 Csatlakozás dátuma: 2018.12.08. Legújabb bejegyzésekJ W, módosítva 3 év-val korábban at 2020.11.24. 15:02
Created 3 év ago at 2020.11.24. 15:02
RE: Server compromised, expect downtime
Bejegyzések: 692 Csatlakozás dátuma: 2020.02.11. Legújabb bejegyzések
Am I reading this correctly, that someone hacked the server?
Why would someone ever do that, lol... there's nothing in that database that isn't visible, it's a public forum.
Why would someone ever do that, lol... there's nothing in that database that isn't visible, it's a public forum.
J W, módosítva 3 év-val korábban at 2020.11.24. 16:21
Created 3 év ago at 2020.11.24. 16:21
RE: Server compromised, expect downtime
Bejegyzések: 692 Csatlakozás dátuma: 2020.02.11. Legújabb bejegyzések
I guess the only thing would be passwords (hopefully they were encrypted).
Guess we should all be watching our accounts and report here if anyone gets hacked.
Guess we should all be watching our accounts and report here if anyone gets hacked.
Z , módosítva 3 év-val korábban at 2020.11.24. 17:14
Created 3 év ago at 2020.11.24. 17:13
RE: Server compromised, expect downtime
Bejegyzések: 201 Csatlakozás dátuma: 2018.03.16. Legújabb bejegyzésekJ W:
I guess the only thing would be passwords (hopefully they were encrypted).
Guess we should all be watching our accounts and report here if anyone gets hacked.
Guess we should all be watching our accounts and report here if anyone gets hacked.
Am I reading this correctly, that someone hacked the server?
Why would someone ever do that, lol... there's nothing in that database that isn't visible, it's a public forum.
Why would someone ever do that, lol... there's nothing in that database that isn't visible, it's a public forum.
Some hackers just like to break in to sites for sport, though they could be after user credentials. Since we don't know yet if credentials were comprimised, if you re-use your DhO account password on any other service/app/site, it'd be a good idea to change your password there.
Z , módosítva 3 év-val korábban at 2020.11.24. 19:52
Created 3 év ago at 2020.11.24. 17:15
RE: Server compromised, expect downtime
Bejegyzések: 201 Csatlakozás dátuma: 2018.03.16. Legújabb bejegyzésekAngel Roberto Puente:
Chris Marti:
Were any of our credentials compromised?
Haha, was the hacker able to access any jhanas?
Siavash ', módosítva 3 év-val korábban at 2020.11.24. 17:17
Created 3 év ago at 2020.11.24. 17:17
RE: Server compromised, expect downtime
Bejegyzések: 1697 Csatlakozás dátuma: 2019.05.05. Legújabb bejegyzések
Don't worry! No one would pay for DhO passwords. It should be something that you could sell it, or something that could open the doors to find something that you could sell it, or gain some power on something or someone and etc. DhO data should not be in those categories, normally!
(Passwords are usually hashed and are not easy to retrieve, if you could do it at all.)
(Passwords are usually hashed and are not easy to retrieve, if you could do it at all.)
Papa Che Dusko, módosítva 3 év-val korábban at 2020.11.24. 23:40
Created 3 év ago at 2020.11.24. 23:40
RE: Server compromised, expect downtime
Bejegyzések: 3048 Csatlakozás dátuma: 2020.03.01. Legújabb bejegyzésekZachary:
Angel Roberto Puente:
Chris Marti:
Were any of our credentials compromised?
Haha, was the hacker able to access any jhanas?
LoL
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/smile.gif)
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/smile.gif)
Pawel K, módosítva 3 év-val korábban at 2020.11.25. 0:16
Created 3 év ago at 2020.11.25. 0:16
RE: Server compromised, expect downtime
Bejegyzések: 1172 Csatlakozás dátuma: 2020.02.22. Legújabb bejegyzésekZachary:
Angel Roberto Puente:
Chris Marti:
Were any of our credentials compromised?
Not two, not one, módosítva 3 év-val korábban at 2020.11.25. 2:08
Created 3 év ago at 2020.11.25. 2:08
RE: Server compromised, expect downtime
Bejegyzések: 1047 Csatlakozás dátuma: 2017.07.13. Legújabb bejegyzésekPapa Che Dusko:
Angel Roberto Puente:
Chris Marti:
Were any of our credentials compromised?
LOL
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/smile.gif)
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/smile.gif)
Lol. very good! Also a promising shortcut, if attainments could be gotten rid of that easily.
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/pac_man.gif)
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/pac_man.gif)
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/pac_man.gif)
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/cold.gif)
Chris M, módosítva 3 év-val korábban at 2020.11.25. 7:57
Created 3 év ago at 2020.11.25. 7:57
RE: Server compromised, expect downtime
Bejegyzések: 5406 Csatlakozás dátuma: 2013.01.26. Legújabb bejegyzések
Since we're not getting any detailed information about this incident or just what has been compromised, I would suggest that a DhO password change is in order, especially if you use your DhO password for anything else.
Pawel K, módosítva 3 év-val korábban at 2020.11.25. 8:15
Created 3 év ago at 2020.11.25. 8:15
RE: Server compromised, expect downtime
Bejegyzések: 1172 Csatlakozás dátuma: 2020.02.22. Legújabb bejegyzésekChris Marti:
Since we're not getting any detailed information about this incident or just what has been compromised, I would suggest that a DhO password change is in order, especially if you use your DhO password for anything else.
![emoticon](https://www.dharmaoverground.org/o/classic-theme/images/emoticons/huh.gif)
Chris M, módosítva 3 év-val korábban at 2020.11.25. 8:19
Created 3 év ago at 2020.11.25. 8:19
RE: Server compromised, expect downtime
Bejegyzések: 5406 Csatlakozás dátuma: 2013.01.26. Legújabb bejegyzésekLewis James, módosítva 3 év-val korábban at 2020.11.25. 11:05
Created 3 év ago at 2020.11.25. 11:05
RE: Server compromised, expect downtime
Bejegyzések: 155 Csatlakozás dátuma: 2015.05.13. Legújabb bejegyzések
Once a software vulnerability is made public and widely available people will set up bots to scan through sites looking for known insecure versions and compromising them / injecting a backdoor / etc on the off-chance they find something valuable.
Looking at the HTTP response headers, the site is running Liferay CE 6.2 GA2 (6.2.1). You can find a list of known vulnerabilities on the Liferay site here:
https://portal.liferay.dev/learn/security/known-vulnerabilities/-/categories/113763930?p_r_p_categoryId=113763930
A bunch of cross-site scripting issues, which can range from fairly minor (think: adding a script to the page that says "Daniel Ingram is a doodoohead"... whether that's minor or not depends on who you are I guess) to pretty annoying (think: adding a bitcoin miner in the background of the page, or redirecting you to a phishing site). The attacker generally won't gain access to the *server*, but may be able to run arbritary scripts in the user's browser. Thankfully if you're using a modern browser there are plenty of security features to stop them from doing anything too bad.
However I would recommend everyone download and use the NoScript browser extension for now and enable it for this site in case any dodgy code has been injected into the site.
There is also a known RCE (remote command execution) exploit for Liferay < 6.2.5, though it seems the details aren't completely public so I don't know if this is a risk, or if so, how much of a risk it is:
https://www.cvedetails.com/cve/CVE-2019-16891/
This kind of exploit can allow an attacker to run whatever program they like on the actual server hardware, which could be a problem and may compromise database info. Passwords should be hashed with, according to the Liferay docs, "PBKDF2--WithHmac--SHA1/160/128000" which basically means passwords are encrypted by a random function run an arbritrary number of times, and encoded using a specified SHA algorithm. However, if that algorithm is SHA1, that has known weaknesses and its use is strongly discouraged these days. Additionally, a note on the Liferay docs says that if a site is upgraded from a prior version to 6.2, they may enable "legacy" password support, which likely uses plain SHA1, which would be even weaker.
Basically in the event that an attacker gained database access they wouldn't get your passwords, but may get hashed versions which may be possible to either brute force (less likely) or look up in a table of known password hashes (if it's SHA1, and you're using a password that can be found in such a table, either using a common word or a password that's previously been leaked, that's quite likely).
Without more info from the hosting provider and/or Daniel's tech team it's hard to say what the actual risk is, though they may not want to disclose that information until it's secured to prevent copycat attacks.
Looking at the HTTP response headers, the site is running Liferay CE 6.2 GA2 (6.2.1). You can find a list of known vulnerabilities on the Liferay site here:
https://portal.liferay.dev/learn/security/known-vulnerabilities/-/categories/113763930?p_r_p_categoryId=113763930
A bunch of cross-site scripting issues, which can range from fairly minor (think: adding a script to the page that says "Daniel Ingram is a doodoohead"... whether that's minor or not depends on who you are I guess) to pretty annoying (think: adding a bitcoin miner in the background of the page, or redirecting you to a phishing site). The attacker generally won't gain access to the *server*, but may be able to run arbritary scripts in the user's browser. Thankfully if you're using a modern browser there are plenty of security features to stop them from doing anything too bad.
However I would recommend everyone download and use the NoScript browser extension for now and enable it for this site in case any dodgy code has been injected into the site.
There is also a known RCE (remote command execution) exploit for Liferay < 6.2.5, though it seems the details aren't completely public so I don't know if this is a risk, or if so, how much of a risk it is:
https://www.cvedetails.com/cve/CVE-2019-16891/
This kind of exploit can allow an attacker to run whatever program they like on the actual server hardware, which could be a problem and may compromise database info. Passwords should be hashed with, according to the Liferay docs, "PBKDF2--WithHmac--SHA1/160/128000" which basically means passwords are encrypted by a random function run an arbritrary number of times, and encoded using a specified SHA algorithm. However, if that algorithm is SHA1, that has known weaknesses and its use is strongly discouraged these days. Additionally, a note on the Liferay docs says that if a site is upgraded from a prior version to 6.2, they may enable "legacy" password support, which likely uses plain SHA1, which would be even weaker.
Basically in the event that an attacker gained database access they wouldn't get your passwords, but may get hashed versions which may be possible to either brute force (less likely) or look up in a table of known password hashes (if it's SHA1, and you're using a password that can be found in such a table, either using a common word or a password that's previously been leaked, that's quite likely).
Without more info from the hosting provider and/or Daniel's tech team it's hard to say what the actual risk is, though they may not want to disclose that information until it's secured to prevent copycat attacks.
Siavash ', módosítva 3 év-val korábban at 2020.11.25. 15:48
Created 3 év ago at 2020.11.25. 15:48
RE: Server compromised, expect downtime
Bejegyzések: 1697 Csatlakozás dátuma: 2019.05.05. Legújabb bejegyzésekChris Marti:
Since we're not getting any detailed information about this incident or just what has been compromised, I would suggest that a DhO password change is in order, especially if you use your DhO password for anything else.
If someone is using their DhO password in other websites/apps/services, I think the thing they should do is to change their passwords on those websites/apps/services, and more importantly be sure that they have a recovery method for their passwords in those systems that they can use if they lost their passwords. Changing DhO password, I don't think is very helpful. The passwords in the database are likely safe.
If someones has access to parts of the server (I mean if they still have access), reading and monitoring the input data on a password change generally should be easier than retrieving the current passwords stored in the database.
Chris M, módosítva 3 év-val korábban at 2020.11.25. 16:10
Created 3 év ago at 2020.11.25. 16:08
RE: Server compromised, expect downtime
Bejegyzések: 5406 Csatlakozás dátuma: 2013.01.26. Legújabb bejegyzésekChanging DhO password, I don't think is very helpful.
Yes, that's right. I've changed my DhO password even though I use a strong password generator (1Password) for all my passwords. I plan to change it often until we know what's up. Also, I just don't trust that anything is "probably safe." I have found that skepticism surrounding cybersecurity issues is a good way to proceed.
Siavash ', módosítva 3 év-val korábban at 2020.11.25. 16:11
Created 3 év ago at 2020.11.25. 16:11
RE: Server compromised, expect downtime
Bejegyzések: 1697 Csatlakozás dátuma: 2019.05.05. Legújabb bejegyzések I use a strong password generator (<This_is_part_of_the_credentials>) for all my passwords.
Mentioning what you use for your password generation would change my options if I wanted to attack you! ;)
Siavash ', módosítva 3 év-val korábban at 2020.11.25. 16:17
Created 3 év ago at 2020.11.25. 16:13
RE: Server compromised, expect downtime
Bejegyzések: 1697 Csatlakozás dátuma: 2019.05.05. Legújabb bejegyzésekAlso, I just don't trust that anything is "probably safe." I have found that skepticism surrounding cybersecurity issues is a good way to proceed.
Just in the context of DhO. If it was a bank for example, that would be a completely different story. Also if there are personal interests/conflicts, that can change it too. I am just emphasizing priorities for an attacker and for the server.
Pawel K, módosítva 3 év-val korábban at 2020.11.25. 16:54
Created 3 év ago at 2020.11.25. 16:54
RE: Server compromised, expect downtime
Bejegyzések: 1172 Csatlakozás dátuma: 2020.02.22. Legújabb bejegyzésekSiavash:
If someones has access to parts of the server (I mean if they still have access), reading and monitoring the input data on a password change generally should be easier than retrieving the current passwords stored in the database.
I do not worry about my account too much. It is not my self and is impermanent anyway.
Siavash ', módosítva 3 év-val korábban at 2020.11.25. 18:34
Created 3 év ago at 2020.11.25. 16:58
RE: Server compromised, expect downtime
Bejegyzések: 1697 Csatlakozás dátuma: 2019.05.05. Legújabb bejegyzésekThat is why I patiently wait until the white borderless box disappear.
Yes. I think this is the right thing to do for most DhO members on DhO currently (Assuming they do the basics: Using firewalls, updating their operating system, updating their browser to its latest version, having security settings on browser at standard or ahigher level, using browser' safety checks, checking ssl lock icon on their browser address bar, not sharing same passwords among different apps...). And considering that DhO uses Single SignOn, and most users do not login each time they use DhO, and instead of actual password, a token is sent to the server that is stored by the browser and the operating system.
Chris M, módosítva 3 év-val korábban at 2020.11.26. 9:01
Created 3 év ago at 2020.11.26. 9:01
RE: Server compromised, expect downtime
Bejegyzések: 5406 Csatlakozás dátuma: 2013.01.26. Legújabb bejegyzésekMentioning what you use for your password generation would change my options if I wanted to attack you! ;)
Sure.
Daniel M Ingram, módosítva 3 év-val korábban at 2020.11.29. 11:40
Created 3 év ago at 2020.11.29. 11:40
RE: Server compromised, expect downtime
Bejegyzések: 3278 Csatlakozás dátuma: 2009.04.20. Legújabb bejegyzések
Simon The Oak has patched what we think was the problem, installed software to scan for files and programs that shouldn't be there, but, still, this version of Liferay is old and has known vulnerabilities, as pointed out above, so Manish is currently trying to upgrade the platform: may his noble efforts go well!
Many thanks to the volunteer support team!
I don't think at this point that the DhO itself was compromised, but that they managed to get access to the server through it and use it to attack other sites, which is still a serious problem, as Hetzner, where we are currently hosted, has a very low tolerance for that sort of thing, so the primary threat to the DhO was them shutting the server down, I think. More updates as they become available.
Many thanks to the volunteer support team!
I don't think at this point that the DhO itself was compromised, but that they managed to get access to the server through it and use it to attack other sites, which is still a serious problem, as Hetzner, where we are currently hosted, has a very low tolerance for that sort of thing, so the primary threat to the DhO was them shutting the server down, I think. More updates as they become available.