Documentation and don't-know-where-it-belongs issues
siwecos / hshs-domxss-scanner Goto Github PK
View Code? Open in Web Editor NEWThis project forked from hackmanit/http-secure-header-scanner
License: MIT License
This project forked from hackmanit/http-secure-header-scanner
License: MIT License
Enhance the scanner to search for well-known javascript contexts like onclick=""
HTML attributes to increase coverage.
Please check this:
$ curl -X HEAD -v https://sekund.eu/ 2>&1 | grep options | xd
00000: 3c 20 78 2d 66 72 61 6d - 65 2d 6f 70 74 69 6f 6e [< x-frame-option]
00010: 73 3a 20 53 41 4d 45 4f - 52 49 47 49 4e 2c 20 53 [s: SAMEORIGIN, S]
00020: 41 4d 45 4f 52 49 47 49 - 4e 0d 0a 3c 20 78 2d 66 [AMEORIGIN..< x-f]
00030: 72 61 6d 65 2d 6f 70 74 - 69 6f 6e 73 3a 20 53 41 [rame-options: SA]
00040: 4d 45 4f 52 49 47 49 4e - 2c 20 53 41 4d 45 4f 52 [MEORIGIN, SAMEOR]
00050: 49 47 49 4e 0d 0a 3c 20 - 78 2d 63 6f 6e 74 65 6e [IGIN..< x-conten]
00060: 74 2d 74 79 70 65 2d 6f - 70 74 69 6f 6e 73 3a 20 [t-type-options: ]
00070: 84 6e 6f 73 6e 69 66 66 - 93 0d 0a .. .. .. .. .. [.nosniff... ]
There are more header lines containing 0x84 0x93 instead of single quets (which would be wrong anyway I guess).
The issue here is: HEADER scanner seems to crash as we get a timeout for it.
Will check now the logs.
At the moment only the Content-Type
Header gets also checked via meta
tag.
We should enable to scan also the meta tags set on the HTML itself, although it might not be as secure as the HTTP-Header.
As we are scanning headers for a secure configuration, we should implement a Cookie-Rating for the Set-Cookie
headers.
Although we don't simulate logins etc. at the moment, we could check if cookies are set when visiting the site and if they have the secure
flag.
If the flag is missing, the rating get's a score of 0.
If the flag is set, a score of 90 is set.
If the flag is set and gets the secure
and the HttpOnly
flag, a score of 100 is set.
If no cookies are set, the scanType
can be hidden
so the test is not shown.
If cookies are set, the scanType
should be warning
.
Don't scan the whole site context to reduce alerts, but limited the regex search to the script
context.
Der Standard zum Request-Interface wurde um den Parameter userAgent
erweitert, siehe hier: https://github.com/SIWECOS/siwecos-core-api/blob/develop/SCANNER_INTERFACE-DOCS.md#request-interface
Bitte überprüfen, ob der Scanner diesen Standard einhält, ggf. Implementieren und dann Issue hier schließen.
Vielen Dank.
Your scanner seems to cache data
if ($this->client === null) {
/**
* The $stack enables caching for the network traffic
* BEST THANKS AND WISHES TO @Kevinrob for guzzle-cache-middleware
*/
$stack = HandlerStack::create();
$stack->push(
new CacheMiddleware(
new PrivateCacheStrategy(
new LaravelCacheStorage(
Cache::store(env('CACHE_DRIVER', 'file'))
)
)
),
'cache'
);
$this->client = new Client(['handler' => $stack]);
}
The consequence is: My skeeve.selfhost.eu, despite delivering correct charset info in the header is said to not deliver it while at the same time, the very same host registered via ip is said to be okay.
This is more than confusing and not in any way helpful when trying to secure a website.
At the moment, also <script>
tags that are within an HTML comment will be found and analyzed.
Waiting for response here: voku/simple_html_dom#20
After merging the current dev to master, no valid JSON response is given by the HEADER Scanner, DOMXSS works fine. @Lednerb
Allow default-src: self
to rate with 100
Zukünftig wird seitens der Core-API ein User-Agent übermittelt.
Die Core-API sendet folgenden Input an die Scanner-URL:
{
"url": "https://siwecos.de",
"callbackurls": [
"http://localhost:9002"
],
"dangerLevel": 0,
"userAgent": "USER_AGENT_ZUR_VERWENDUNG"
}
Dieser userAgent
soll zum Aufrufen der zu scannenden Seite verwendet werden.
Sollte userAgent
nicht gesetzt sein, so sollte der Scanner einen Standard verwenden, um verschiedene WAFs oder andere Sperrmaßnahmen umgehen und einen Scan erfolgreich durchführen zu können.
Ein Standard könnte bspw. folgender sein:
Mozilla/5.0 (X11; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0
Bitte Implementieren.
Siehe auch:
SIWECOS/siwecos-core-api#158
Implement a check for the Expect-CT
Header.
Further Information:
https://www.kuketz-blog.de/certificate-transparency-expect-ct-http-security-header/
https://scotthelme.co.uk/a-new-security-header-expect-ct/
Momentan wird beim ContentTypeRating
auch auf eingebundene meta
-Tags geachtet.
Allerdings wird jeweils nur der erste Treffer ausgegeben und bewertet.
Es wäre sinnvoll, dies so umzugestalten, dass es Punktabzug im Rating sowie eine Fehlermeldung gibt, wenn das bzw. die meta
-Tags mehrmals gesetzt wurden.
Ebenfalls dann auch in #37 beachten.
Ähnlich wie bei #67
A scan of https://siwecos.de shows below result
So no sinks found and no sources found. Nevertheless a score of 0.
( commit 2f8539f )
Scanning my domain gives this text which is unclear to me:
Inkorrekte HTTP Content-Type Konfiguration
Der Content-Type ist eine Angabe, die für gewöhnlich im Kopfbereich der Webseite, dem sogenannten Header, untergebracht wird. Durch diese Angaben wird der Zeichensatz und der Typ des Inhalts der Seite definiert. Sollte eine Definition fehlen, wird der Webbrowser versuchen den Content-Type zu erraten; dies kann zu Sicherheitslücken wie Code-Page-Sniffing führen. Diese Angaben sind zudem wichtig, damit die Webseite in jedem Browser und auf jedem Computer einwandfrei dargestellt wird. Wenn ein Server ein Dokument an einen User-Agent sendet (zum Beispiel zum Browser) ist es nützlich, im Content-Type-Feld des HTTP-Header die Art des Dateiformates zu hinterlegen. Diese Informationen deklarieren den MIME-Typ und senden entsprechend die Zeichenkodierung des Dokuments wie text/html, text/plain, etc. an den Browser.
Mehr Informationen
Der Header "text/html; charset=utf-8" ist korrekt gesetzt und entspricht den Empfehlungen.
Die Angabe im "<meta charset="utf-8" />" im HTML-Header
Die Angabe im "<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">" im HTML-Header
This is the reesult as stored in the database:
{
"hasError" : false,
"tests" : [
{
"hasError" : false,
"scoreType" : "warning",
"score" : 100,
"name" : "CONTENT_SECURITY_POLICY",
"testDetails" : [
{
"placeholder" : "CSP_CORRECT",
"values" : {
"HEADER" : "default-src 'none'; frame-src 'self'; font-src 'self'; img-src 'self' siwecos.de; script-src 'self'; style-src 'self';"
}
}
],
"errorMessage" : null
},
{
"hasError" : false,
"scoreType" : "warning",
"name" : "CONTENT_TYPE",
"testDetails" : [
{
"placeholder" : "CT_CORRECT",
"values" : {
"HEADER" : "text/html; charset=utf-8"
}
},
{
"values" : {
"META" : "<meta charset=\"utf-8\" />"
},
"placeholder" : "CT_META_TAG_SET_CORRECT"
},
{
"placeholder" : "CT_META_TAG_SET_CORRECT",
"values" : {
"META" : "<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\">"
}
}
],
"errorMessage" : null,
"score" : 60
},
{
"score" : 100,
"testDetails" : [
{
"placeholder" : "HPKP_MORE_15",
"values" : {
"HEADER" : "pin-sha256=\"base64+primary==\"; pin-sha256=\"base64+backup==\"; max-age=5184000; includeSubDomains"
}
},
{
"values" : {
"HEADER" : "pin-sha256=\"base64+primary==\"; pin-sha256=\"base64+backup==\"; max-age=5184000; includeSubDomains"
},
"placeholder" : "INCLUDE_SUBDOMAINS"
}
],
"name" : "PUBLIC_KEY_PINS",
"errorMessage" : null,
"hasError" : false,
"scoreType" : "bonus"
},
{
"score" : 100,
"testDetails" : [
{
"values" : {
"HEADER" : "max-age=31536000; includeSubDomains"
},
"placeholder" : "HSTS_MORE_6"
},
{
"placeholder" : "INCLUDE_SUBDOMAINS",
"values" : {
"HEADER" : "max-age=31536000; includeSubDomains"
}
}
],
"name" : "STRICT_TRANSPORT_SECURITY",
"errorMessage" : null,
"hasError" : false,
"scoreType" : "warning"
},
{
"score" : 100,
"testDetails" : [
{
"placeholder" : "XCTO_CORRECT",
"values" : {
"HEADER" : "nosniff"
}
}
],
"name" : "X_CONTENT_TYPE_OPTIONS",
"errorMessage" : null,
"hasError" : false,
"scoreType" : "warning"
},
{
"errorMessage" : null,
"testDetails" : [
{
"values" : {
"HEADER" : "SAMEORIGIN"
},
"placeholder" : "XFO_CORRECT"
}
],
"name" : "X_FRAME_OPTIONS",
"score" : 100,
"scoreType" : "warning",
"hasError" : false
},
{
"hasError" : false,
"scoreType" : "warning",
"score" : 100,
"name" : "X_XSS_PROTECTION",
"testDetails" : [
{
"placeholder" : "XXSS_CORRECT",
"values" : {
"HEADER" : "1; mode=block"
}
},
{
"placeholder" : "XXSS_BLOCK",
"values" : {
"HEADER" : "1; mode=block"
}
}
],
"errorMessage" : null
}
],
"score" : 93,
"errorMessage" : null,
"name" : "HEADER"
}
You see: All testdetails for CONTENT_TYPE are '*_CORRECT'. Yet the score is only 60,
/** @test */
public function if_there_is_an_http_error_the_correct_formatted_error_message_will_be_send()
{
$response = $this->json('POST', '/api/v1/domxss', [
'url' => 'https://url-but-not-available.info',
]);
testet die Antwort, wenn eine URL nicht verfügbar ist.
Im Test wird dabei jedoch direkt versucht die URL aufzurufen und Fehlermeldungen ausgegeben.
Externe Seitenaufrufe sollten generell in Tests gemocked werden, siehe bspw. Tests in der aktuellen siwecos-business-layer Version.
Sofern sich die Möglichkeit ergibt, sollte das geändert werden, einmal im HSHS- und einmal im DOMXSS-Teil.
Momentan werden für ein Cookie mehrere Ausgaben generiert:
{
"name": "SET_COOKIE",
"hasError": false,
"errorMessage": null,
"score": 10,
"scoreType": "warning",
"testDetails": [
{
"placeholder": "NO_SECURE_FLAG_SET",
"values": {
"COOKIE": "__cfduid=dbad1f29600e6c6e83d13278b9cce87331554355699; expires=Fri, 03-Apr-20 05:28:19 GMT; path=/; domain=.laravel.com; HttpOnly"
}
},
{
"placeholder": "HTTPONLY_FLAG_SET",
"values": {
"COOKIE": "__cfduid=dbad1f29600e6c6e83d13278b9cce87331554355699; expires=Fri, 03-Apr-20 05:28:19 GMT; path=/; domain=.laravel.com; HttpOnly"
}
}
]
},
Es wäre von Vorteil, wenn die Text-Ausgabe so angepasst wird, dass die Sprachstrings 3 Placeholder beinhalten können:
COOKIE_NAME
: Name des CookiesSECURE
: Ist secure
-Flag gesetzt oder nichtHTTPONLY
: Ist httpOnly
-Flag gesetzt oder nichtDann wird für jedes gesetztes Cookie lediglich eine Ausgabe erzeugt, anstatt momentan 2.
Es ist uns ein Fehler aufgefallen, der das SetCookieRating
betroffen hatte:
Es wurde ein invalider Header gefunden.
Als fix wurde das neue testDetail
: INVALID_HEADER
eingeführt.
Es sollte überprüft werden, bei welchen Rating
s das noch Sinn macht und ggf. angepasst werden.
Wahrscheinlich macht das in der CSP Sinn etc.
Siehe auch: 9a765f8
Edit:
Und 1fea261
Implement Tests for the Feature-Policy:
Header.
A web platform API which gives a website the ability to allow and deny the use of browser features in its own frame, and in iframes that it embeds.
Browser Support:
https://caniuse.com/#feat=feature-policy
Further information:
https://developers.google.com/web/updates/2018/06/feature-policy
https://docs.google.com/document/d/1k0Ua-ZWlM_PsFCFdLMa8kaVTo32PeNZ4G7FFHqpFx4E/edit#
https://github.com/WICG/feature-policy/blob/master/features.md
Although the Feature-Policy
Header is not standardized and implemented in all web browsers, we can implement a test for this new header.
*
) are used, we should set a score of 0, because that's a bad practice.If possible features are not used anyway, they can get disabled to reduce the potential attack surface.
When an HSHS scan is done and no error getting the site's headers occurred, the global hasError
is set to false
.
If there is an Error at a specific test, for example, the CSPRating
, the test's hasError
is set to true
.
Should the global hasError
also set to true
if one of the included ratings/tests get an error or is the global one reserved for bigger problems?
What do you think @SniperSister @Weegy @Skeeve ?
Please see below (part of) the result of a scan of http://www.tom123.de. It's invalid json
{
"name": "PUBLIC_KEY_PINS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "bonus",
"testDetails": [{
"placeholder": "HEADER",
"values": ["pin-sha256=\\"
base64 + primary == \\"; pin-sha256=\\"
base64 + backup == \\"; max-age=5184000; includeSubDomains"
]
}, {
"placeholder": "HPKP_MORE_15"
}, {
"placeholder": "INCLUDE_SUBDOMAINS"
}]
}
Unformated complete string below:
[{"name":"CONTENT_SECURITY_POLICY","hasError":false,"errorMessage":null,"score":100,"scoreType":"warning","testDetails":[{"placeholder":"HEADER","values":["default-src 'none'; script-src 'self'; connect-src 'self'; img-src 'self'; style-src 'self';"]},{"placeholder":"CSP_CORRECT"}]},{"name":"CONTENT_TYPE","hasError":false,"errorMessage":null,"score":100,"scoreType":"warning","testDetails":[{"placeholder":"HEADER","values":["text\\/html; charset=utf-8"]},{"placeholder":"CT_CORRECT"}]},{"name":"PUBLIC_KEY_PINS","hasError":false,"errorMessage":null,"score":100,"scoreType":"bonus","testDetails":[{"placeholder":"HEADER","values":["pin-sha256=\\"base64+primary==\\"; pin-sha256=\\"base64+backup==\\"; max-age=5184000; includeSubDomains"]},{"placeholder":"HPKP_MORE_15"},{"placeholder":"INCLUDE_SUBDOMAINS"}]},{"name":"STRICT_TRANSPORT_SECURITY","hasError":false,"errorMessage":null,"score":100,"scoreType":"warning","testDetails":[{"placeholder":"HEADER","values":["max-age=31536000; includeSubDomains"]},{"placeholder":"HSTS_MORE_6"},{"placeholder":"INCLUDE_SUBDOMAINS"}]},{"name":"X_CONTENT_TYPE_OPTIONS","hasError":false,"errorMessage":null,"score":100,"scoreType":"warning","testDetails":[{"placeholder":"HEADER","values":["nosniff"]},{"placeholder":"XCTO_CORRECT"}]},{"name":"X_FRAME_OPTIONS","hasError":false,"errorMessage":null,"score":100,"scoreType":"warning","testDetails":[{"placeholder":"HEADER","values":["SAMEORIGIN"]},{"placeholder":"XFO_CORRECT"}]},{"name":"X_XSS_PROTECTION","hasError":false,"errorMessage":null,"score":100,"scoreType":"warning","testDetails":[{"placeholder":"HEADER","values":["1; mode=block"]},{"placeholder":"XXSS_CORRECT"},{"placeholder":"XXSS_BLOCK"}]}]
@Weegy please group and name your issues in a better way, so there are not 15 commits with the same comment in the future.
Also please test your Fixes so no new errors occur:
HSHS-DOMXSS-Scanner/app/Ratings/XFrameOptionsRating.php
Lines 25 to 26 in aedb7ac
The placeholder
value is wrong;
Why does the XFrameOptionsRating
not include the original HEADER
?
Also some Ratings include the $header
and some not - and not consistent.
Doesn't checked the other ones for further errors.
In general:
Why does it make sense for you to send the header for every test again and not send the header once...
I don't get it - why does each header test cannot simply contain the actually set header so the siwecos user can see the actual set header and the best practice one side by side in the frontend?
Please provide us with values for the placeholders your scanner uses.
The list of placeholders I currently know of is
DOMXSS:
NO_SINKS_FOUND
NO_SOURCES_FOUND
SINKS_FOUND
SOURCES_FOUND
HEADER:
CSP_CORRECT
CSP_NO_UNSAFE_INCLUDED
CSP_UNSAFE_INCLUDED
CT_CORRECT
CT_HEADER_WITHOUT_CHARSET
CT_META_TAG_SET
HSTS_LESS_6
HSTS_MORE_6
HSTS_PRELOAD
INCLUDE_SUBDOMAINS
XCTO_CORRECT
XFO_CORRECT
XXSS_BLOCK
XXSS_CORRECT
Please add all placeholders I missed.
When providing us with the (german) text, please take your test results into account.
Please just send it as a plain text file or a fresh excel sheet - whatever you prefer. We will put it into our central gettext-file.
We should implement a Version number for all scanners and the API so we can quickly see which version is used (or deployed).
Hi!
Nach Deployment gibt der DOMXSS beim Aufruf HTTP 500 zurück.
Achtung! Deployed wurde in der Cloud. Bitte @Weegy nach Details fragen.
When trying to solve the CSP issues reported by my initial scans, I made Header and DOMXSS scanner crash it seems.
As my "domain" is not always online, please see the curl output here:
$ curl -v http://skeeve.selfhost.eu
* Rebuilt URL to: http://skeeve.selfhost.eu/
* Trying 78.49.227.218...
* TCP_NODELAY set
* Connected to skeeve.selfhost.eu (78.49.227.218) port 80 (#0)
> GET / HTTP/1.1
> Host: skeeve.selfhost.eu
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Wed, 21 Mar 2018 11:19:19 GMT
< Server: Apache/2.4.28 (Unix)
< Last-Modified: Wed, 21 Mar 2018 11:07:37 GMT
< ETag: "14d-567ea32de7040"
< Accept-Ranges: bytes
< Content-Length: 333
< Content-Security-Policy: default-src 'self'
< X-Content-Security-Policy: default-src 'self'
< X-WebKit-CSP: default-src 'self'
< Content-Type: text/html
<
<html>
<head>
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'">
<meta http-equiv="X-Content-Security-Policy" content="default-src 'self'; script-src 'self'">
<meta http-equiv="X-WebKit-CSP" content="default-src 'self'; script-src 'self'">
</head>
<body><h1>It works!</h1></body></html>
* Connection #0 to host skeeve.selfhost.eu left intact
Es könnte sein, dass wir mit der Umstellung der Queue in ein Rate-Limit des Scanners laufen.
Sollte in der API - Route komplett entfernt werden.
Hi,
die aktuelle Live Version erkennt laut Testdetail Meldungen bei dieser Seite hier einen mehrfach gesetzten x-frame-options header, de-facto wird aber garkeiner gesetzt:
https://www.dr-menzel-it.de/
scoreType: hidden
da zu kritisch und für Zielgruppe irrelevant.
Besides the Content-Security-Policy
header, there is the Content-Security-Policy-Report-Only
header that is used to report violations against the CSP without blocking content.
We should implement a check for this header and send a hint to the user, that he should consider using the Content-Security-Header
instead.
Please see your testdetails:
[{
"name": "CONTENT_SECURITY_POLICY",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["default-src 'none'; script-src 'self'; connect-src 'self'; img-src 'self'; style-src 'self';"]
}, {
"placeholder": "CSP_CORRECT"
}]
}, {
"name": "CONTENT_TYPE",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["text\\/html; charset=utf-8"]
}, {
"placeholder": "CT_CORRECT"
}]
}, {
"name": "PUBLIC_KEY_PINS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "bonus",
"testDetails": [{
"placeholder": "HEADER",
"values": ["pin-sha256=\\"
base64 + primary == \\"; pin-sha256=\\"
base64 + backup == \\"; max-age=5184000; includeSubDomains"
]
}, {
"placeholder": "HPKP_MORE_15"
}, {
"placeholder": "INCLUDE_SUBDOMAINS"
}]
}, {
"name": "STRICT_TRANSPORT_SECURITY",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["max-age=31536000; includeSubDomains"]
}, {
"placeholder": "HSTS_MORE_6"
}, {
"placeholder": "INCLUDE_SUBDOMAINS"
}]
}, {
"name": "X_CONTENT_TYPE_OPTIONS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["nosniff"]
}, {
"placeholder": "XCTO_CORRECT"
}]
}, {
"name": "X_FRAME_OPTIONS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["sameorigin"]
}, {
"placeholder": "XFO_CORRECT"
}]
}, {
"name": "X_XSS_PROTECTION",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["1; mode=block"]
}, {
"placeholder": "XXSS_CORRECT"
}, {
"placeholder": "XXSS_BLOCK"
}]
}]
Please compare your testdetails with the api. "values" has to be an associative array.
Additionally using "HEADER" only makes sense if the text should always be the same for each test.
Header:
Content-Security-Policy: #default-src 'self'; font-src 'self'
The #
sign is illegal,
Browser says:
Content Security Policy: Unbekannte Direktive '#default-src' kann nicht verarbeitet werden
Rating rates:100
FIX:
Only allow a whitelist of directives and values such as self
or none
or URLs, without wildcards.
The current version on production gives wrongly encoded results.
See:
"testDetails":[{"placeholder":"CT_META_TAG_SET_CORRECT","values":{"META":"<meta http-equiv=\\"Content-Type\\" content=\\"text\\/html; charset=UTF-8\\">"}}
Too many backslashes.
formatting with with json_pp throws
, or } expected while parsing object/hash, at character offset 393 (before "Content-Type\\\\" co...") at /usr/bin/json_pp5.18 line 45.
CONTENT_SECURITY_POLICY wirft auf meiner Beispielseite einen Critical Issue mit Score 0, weil ich inline JS erlaube - ich verstehe dass das für die CSP wichtig ist und die Schutzwirkung massiv einschränkt, aber:
Daher Vorschlag:
Keine CSP: ScoreType Warning, Score 0
CSP mit unsafe: ScoreType Info, Score 50
CSP ohne unsafe: alles gut, Score 100
HPKP ist bei unserer Zielgruppe eine ziemlich gefährliche Nummer, ich würde daher vorschlagen den auf "Bonus" zu setzen und damit nur anzuzeigen (und im Rating einzubeziehen) wenn er gesetzt ist - wenn HPKP fehlt, sollte SIWECOS sonst einfach ignorieren.
Excerpt from a HEADER Scan result:
"version" : "1.0.0",
"name" : "HEADER",
"score" : 75,
"errorMessage" : null,
"hasError" : true
This leads to BLA failing here:
if ($item['has_error']) {
$errorRaw = $item['complete_request']['errorMessage'];
$error = array();
$error['report'] = html_entity_decode(__('siwecos.' . $errorRaw['placeholder']));
$error['has_error'] = true;
$error['score'] = 0;
if (array_key_exists('values', $errorRaw)) { // <- HERE
No matter what we put into the sunbros.de head, the test complains about missing charset.
We tried:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
as well as
<meta charset="UTF-8" />
This is the result:
[{
"name": "CONTENT_SECURITY_POLICY",
"hasError": false,
"errorMessage": null,
"score": 50,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["default-src 'self';"]
}, {
"placeholder": "CSP_NO_UNSAFE_INCLUDED"
}]
}, {
"name": "CONTENT_TYPE",
"hasError": false,
"errorMessage": null,
"score": 60,
"scoreType": "warning",
"testDetails": [{
"placeholder": "META",
"values": ["<meta http-equiv=\"Content-Type\" content=\"text\/html; charset=utf-8\" \/>"]
}, {
"placeholder": "CT_META_TAG_SET_CORRECT"
}, {
"placeholder": "HEADER",
"values": ["text\/html"]
}, {
"placeholder": "CT_HEADER_WITHOUT_CHARSET"
}]
}, {
"name": "PUBLIC_KEY_PINS",
"hasError": true,
"errorMessage": "HEADER_NOT_SET",
"score": 0,
"scoreType": "info",
"testDetails": []
}, {
"name": "STRICT_TRANSPORT_SECURITY",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["max-age=31536000; includeSubDomains"]
}, {
"placeholder": "HSTS_MORE_6"
}, {
"placeholder": "INCLUDE_SUBDOMAINS"
}]
}, {
"name": "X_CONTENT_TYPE_OPTIONS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["nosniff"]
}, {
"placeholder": "XCTO_CORRECT"
}]
}, {
"name": "X_FRAME_OPTIONS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["sameorigin"]
}, {
"placeholder": "XFO_CORRECT"
}]
}, {
"name": "X_XSS_PROTECTION",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["1; mode=block"]
}, {
"placeholder": "XXSS_CORRECT"
}, {
"placeholder": "XXSS_BLOCK"
}]
}]
On siwecos.de there is no complaint about the charset even though we only have
<meta charset="UTF-8" />
Hi all,
I checked the new SIWECOS site with the scanner today and got a degraded score from the header scanner because of having "unsafe-inline" as part of the style-src property.
Unsafe-Inline however is only in there for legacy browser support, modern browsers will detect the sha256 hash that the site provides and ignore unsafe-inline or unsafe-eval.
The combination of unsafe-inline (used as fallback) and hash/nonce is the only way of using inline scripts or styles with CSP and having a wide range of browsers supported. I'm aware that such a combination still leaves a tiny loophole, but considering the very low market share of these legacy browser I would like to raise a discussion if this setup isn't "secure enough" to deserve a 100 score.
As I'm not allowed to open #7…
This is the current curl output:
$ curl -v https://sunbros.de/ > /dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 5.35.226.237...
* TCP_NODELAY set
* Connected to sunbros.de (5.35.226.237) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [106 bytes data]
* NPN, negotiated HTTP1.1
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2478 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Unknown (67):
} [36 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=sunbros.de
* start date: Jan 2 08:55:47 2018 GMT
* expire date: Apr 2 08:55:47 2018 GMT
* subjectAltName: host "sunbros.de" matched cert's "sunbros.de"
* issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
* SSL certificate verify ok.
> GET / HTTP/1.1
> Host: sunbros.de
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Wed, 21 Mar 2018 12:17:36 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 544
< Connection: keep-alive
< Server: Apache
< Last-Modified: Wed, 21 Mar 2018 10:56:34 GMT
< ETag: "220-567ea0b6229b0"
< Accept-Ranges: bytes
< X-Frame-Options: sameorigin
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Referrer-Policy: strict-origin
< Content-Security-Policy: default-src 'self';
<
{ [544 bytes data]
100 544 100 544 0 0 2025 0 --:--:-- --:--:-- --:--:-- 2029
* Connection #0 to host sunbros.de left intact
so Content-Type: text/html; charset=utf-8
is present.
Still there is the Header Complaint
And this is in the database:
[{
"name": "CONTENT_SECURITY_POLICY",
"hasError": false,
"errorMessage": null,
"score": 50,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["default-src 'self';"]
}, {
"placeholder": "CSP_NO_UNSAFE_INCLUDED"
}]
}, {
"name": "CONTENT_TYPE",
"hasError": false,
"errorMessage": null,
"score": 60,
"scoreType": "warning",
"testDetails": [{
"placeholder": "META",
"values": ["<meta http-equiv=\"Content-Type\" content=\"text\/html; charset=utf-8\" \/>"]
}, {
"placeholder": "CT_META_TAG_SET_CORRECT"
}, {
"placeholder": "HEADER",
"values": ["text\/html"]
}, {
"placeholder": "CT_HEADER_WITHOUT_CHARSET"
}]
}, {
"name": "PUBLIC_KEY_PINS",
"hasError": true,
"errorMessage": "HEADER_NOT_SET",
"score": 0,
"scoreType": "info",
"testDetails": []
}, {
"name": "STRICT_TRANSPORT_SECURITY",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["max-age=31536000; includeSubDomains"]
}, {
"placeholder": "HSTS_MORE_6"
}, {
"placeholder": "INCLUDE_SUBDOMAINS"
}]
}, {
"name": "X_CONTENT_TYPE_OPTIONS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["nosniff"]
}, {
"placeholder": "XCTO_CORRECT"
}]
}, {
"name": "X_FRAME_OPTIONS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["sameorigin"]
}, {
"placeholder": "XFO_CORRECT"
}]
}, {
"name": "X_XSS_PROTECTION",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HEADER",
"values": ["1; mode=block"]
}, {
"placeholder": "XXSS_CORRECT"
}, {
"placeholder": "XXSS_BLOCK"
}]
}]
Please check why the header scanner fails with the botfrei domains (botfrei.de and botfrei.eu)
Enhance logging and implement a scanner tag in the log messages like: [DOMXSS]
and [HEADER]
Use a real web server instead of the php artisan
command for better performance
It's a good idea to use a PHP-Worker for:
$check = new DomxssCheck($request->url);
$this->notifyCallbacks($request->callbackurls, $check);
Add tests for the Referrer-Policy:
header.
Although it's not directly relevant to the user's security it can increase the user's privacy.
Browser support:
https://caniuse.com/#feat=referrer-policy
Further information:
https://scotthelme.co.uk/a-new-security-header-referrer-policy/
We should implement a test for this header and set it's scoreType
to bonus
or info
or something because it does not affect the security but privacy.
If the header is set so the privcacy of the user is protected the site will get a good rating.
Using the no-refferer
, same-origin
or strict-origin*
values will get a score of 100.
Using the other values will get a score of 0 because the referrer may be leaked on the network and to any other sites.
Previously this read
Everything is 100% except for one bonus and the overall score is 95%. Why?
The questions now merely are
{
"name": "HEADER",
"hasError": false,
"errorMessage": null,
"score": 95,
"tests": [{
"name": "CONTENT_SECURITY_POLICY",
"hasError": false,
"errorMessage": null,
"score": 75,
"scoreType": "warning",
"testDetails": [{
"placeholder": "CSP_NO_UNSAFE_INCLUDED",
"values": [{
"HEADER": "\"default-src 'self'; script-src 'self' 'sha256-GDawD7jO073hEjCl6AdkNG1dWLYFcTlSuyzyFuGqZ0k=' https:\\\/\\\/webstats.eco.de; style-src 'self' https:\\\/\\\/fonts.googleapis.com; img-src 'self' https:\\\/\\\/siwecos.de https:\\\/\\\/webstats.eco.de data:; font-src 'self' https:\\\/\\\/fonts.gstatic.com;\""
}]
}]
}, {
"name": "CONTENT_TYPE",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "CT_META_TAG_SET_CORRECT",
"values": {
"META": "<meta http-equiv=\"Content-Type\" content=\"text\/html; charset=UTF-8\" \/>"
}
}, {
"placeholder": "CT_CORRECT",
"values": {
"HEADER": "text\/html; charset=UTF-8"
}
}]
}, {
"name": "PUBLIC_KEY_PINS",
"hasError": true,
"errorMessage": "HEADER_NOT_SET",
"score": 0,
"scoreType": "bonus",
"testDetails": []
}, {
"name": "STRICT_TRANSPORT_SECURITY",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "HSTS_MORE_6",
"values": []
}, {
"placeholder": "INCLUDE_SUBDOMAINS",
"values": []
}]
}, {
"name": "X_CONTENT_TYPE_OPTIONS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "XCTO_CORRECT",
"values": []
}]
}, {
"name": "X_FRAME_OPTIONS",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "XFO_CORRECT",
"values": []
}]
}, {
"name": "X_XSS_PROTECTION",
"hasError": false,
"errorMessage": null,
"score": 100,
"scoreType": "warning",
"testDetails": [{
"placeholder": "XXSS_CORRECT",
"values": []
}, {
"placeholder": "XXSS_BLOCK",
"values": []
}]
}]
}
This seems illogical to me:
{
"hasError" : true,
"name" : "X_FRAME_OPTIONS",
"score" : 0,
"errorMessage" : "HEADER_NOT_SET",
"scoreType" : "warning",
"testDetails" : [
{
"values" : [],
"placeholder" : "HEADER_SET_MULTIPLE_TIMES"
}
]
},
HEADER_NOT_SET seems to be the exact opposite of HEADER_SET_MULTIPLE_TIMES
Could you please clarify/fix?
Checking with curl (my private .de domain) shows that X-Frame isn't set at all.
The scanner does not respect the correct format for errorMessage
s as it's defined here:
https://app.swaggerhub.com/apis/SIWECOS/siwecos-scanner_api/docs/0.0.1
Implement the messages as TranslateableMessage.
Currently the core api does not receive an error when your scanner runs into issues like the domain is not reachable.
Currently we handle this with a timeout, but this is just a workaround.
Please provide, via callback, an error message to the core api.
An example could be (according to the api):
[{
"name": "ERROR",
"hasError": true,
"dangerlevel": 0,
"errorMessage": {
"placeholder": "NO_RESPONSE",
"values": {
"domain": "http://your.do.main",
}
},
"score": 0,
"scoreType": "success",
"testDetails": []
}]
Check SIWECOS/siwecos-core-api#68 (comment) for more information, especially about example values for the ERROR and NO_RESPONSE placeholders.
Please make sure to use error placeholders which are either specific for your scanner (like DOMXSS_OUT_OF_MEMORY) if it is a scanner specific error message or, if the error is a general one which can happen to any scanner, non-specific (like NO_RESPONSE).
Currently I see in the Header Scanner results the placeholders
META
HEADER
but we do not have values.
Outpu looks like this:
Überprüfung des HTTP Content-Types
Die Content Type Angabe ist korrekt konfiguriert.
Mehr Informationen
* siwecos.META
* Die Header-Angabe ist via `<meta>`-Tag gesetzt und entspricht den Empfehlungen.
* siwecos.HEADER
* Der Header ist korrekt gesetzt und entspricht den Empfehlungen.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.