sixlabors / imagesharp.web Goto Github PK
View Code? Open in Web Editor NEW:globe_with_meridians: High Performance Image Processing Middleware for ASP.NET- Core.
Home Page: https://sixlabors.com/products/imagesharp-web/
License: Other
:globe_with_meridians: High Performance Image Processing Middleware for ASP.NET- Core.
Home Page: https://sixlabors.com/products/imagesharp-web/
License: Other
Repro: Delete the wwwroot
folder, and see this exception
ArgumentNullException: Value cannot be null. Parameter name: path1 System.IO.Path.Combine(string path1, string path2) SixLabors.ImageSharp.Web.Caching.PhysicalFileSystemCache+<SetAsync>d__12.MoveNext()
Ideally the exception should be more descriptive. Or just be no-op without an exception.
Hi,
I ran into caching issues while adding this middleware into my app, all matched urls (which are in the form of /imagepool/3a47ac8a-c95c-4a6c-a96c-94613d3f567a.ext
) returned the same image, the one which was accessed first. Adding a processing parameter caused the correct image being loaded, but again trying to load other images with the same parameters returned the firstly accessed image.
So I took a look at the default CacheHash implementation and to me it looks like the hash is generated from the empty(?) buffer from the pool. Seems the url-value is only used to get the extension and the size of the buffer, thus if the hash is calculated from a buffer of zeroes(?) it only changes if the buffer size changes. After switching to my own implementation of the CacheHash everything worked as expected.
Related to this log:
It's using Error level which means whenever a non-existent image is requested there will be a log message. We think (OC) this should be Information
only.
I don't think we can lower the log level for specific events, and we don't want to discard all errors.
Install ImageSharp.Web-master on a Mac. Start Visual Studio with ImageSharp.Web.sln
Build ImageSharp.Web.Sample (see ticket #53 on errors).
Run it.
The demo page looks quite poor like this
All requests for an image result in an exception like:
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request starting HTTP/1.1 GET http://localhost:2646/imagesharp-logo.png?width=300&bgcolor=FFFF00 �[40m�[32minfo�[39m�[22m�[49m: Microsoft.AspNetCore.Hosting.Internal.WebHost[1] Request starting HTTP/1.1 GET http://localhost:2646/imagesharp-logo.png?width=300&bgcolor=FFFF00 Exception thrown: 'System.MissingMethodException' in SixLabors.ImageSharp.Web.dll Exception thrown: 'System.NotSupportedException' in SixLabors.ImageSharp.Web.dll Exception thrown: 'System.MissingMethodException' in System.Private.CoreLib.dll Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request finished in 4214.771ms 500 text/html; charset=utf-8 �[40m�[32minfo�[39m�[22m�[49m: Microsoft.AspNetCore.Hosting.Internal.WebHost[2] Request finished in 4214.771ms 500 text/html; charset=utf-8 �[40m�[32minfo�[39m�[22m�[49m: Microsoft.AspNetCore.Hosting.Internal.WebHost[1] Request starting HTTP/1.1 GET http://localhost:2646/imagesharp-logo.png?width=300&bgcolor=128,28,32 Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request starting HTTP/1.1 GET http://localhost:2646/imagesharp-logo.png?width=300&bgcolor=128,28,32 �[41m�[30mfail�[39m�[22m�[49m: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1] An unhandled exception has occurred while executing the request. System.MissingMethodException: Method not found: 'Void Microsoft.Net.Http.Headers.EntityTagHeaderValue..ctor(System.String)'. at SixLabors.ImageSharp.Web.Middleware.ImageContext.ComputeLastModified() at SixLabors.ImageSharp.Web.Middleware.ImageContext.ComprehendRequestHeaders(DateTimeOffset lastModified, Int64 length) in .../ImageSharp.Web-master/src/ImageSharp.Web/Middleware/ImageContext.cs:line 103 at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.SendResponse(ImageContext imageContext, String key, Stream stream, ImageMetaData metadata) in .../ImageSharp.Web-master/src/ImageSharp.Web/Middleware/ImageSharpMiddleware.cs:line 279 at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context) in .../ImageSharp.Web-master/src/ImageSharp.Web/Middleware/ImageSharpMiddleware.cs:line 203 at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context) Exception thrown: 'System.MissingMethodException' in SixLabors.ImageSharp.Web.dll Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware:Error: An unhandled exception has occurred while executing the request.
packages/sixlabors.core/1.0.0-beta0006/lib/netcoreapp2.0/SixLabors.Core.dll packages/sixlabors.imagesharp/1.0.0-beta0005/lib/netcoreapp2.1/SixLabors.ImageSharp.dll
DEBUG
and RELEASE
modeUsing the new AzureBlobStorageImageProvider is extremely slow.
services.AddImageSharpCore ()
.SetRequestParser<QueryCollectionRequestParser> ()
.Configure<PhysicalFileSystemCacheOptions> (options => {
options.CacheFolder = "/tmp/pn-cache";
})
.SetCache<PhysicalFileSystemCache> ()
.SetCacheHash<CacheHash> ()
.AddProvider<AzureBlobStorageImageProvider> (AzureProviderFactory)
.Configure<AzureBlobStorageImageProviderOptions> (options => {
options.ConnectionString = connectionString;
options.ContainerName = containerName;
})
.AddProcessor<ResizeWebProcessor> ();
where AzureProviderFactory is
private static AzureBlobStorageImageProvider AzureProviderFactory (IServiceProvider provider) {
var containerName = provider.GetRequiredService<IOptions<ImageFileStorageSettings>> ().Value.ContainerName;
return new AzureBlobStorageImageProvider (
provider.GetRequiredService<IOptions<AzureBlobStorageImageProviderOptions>> (),
provider.GetRequiredService<FormatUtilities> ()) {
Match = context => {
return context.Request.Path.StartsWithSegments ($"/{containerName}");
}
};
}
Ubuntu 19.04 and CoreOS running docker and Kubernetes in Azure Container Service.
I believe the issue is that ImageSharpMiddleware calls
IImageResolver sourceImageResolver = await provider.GetAsync(context).ConfigureAwait(false);
before checking the cache and in the case of the AzureBlobStorageImageProvider this causes a CloudBlockBlob to be instantiated and ExistsAsync() to be called. This kills performance especially when many images are being loaded at once.
On the other hand, I might just be doing it all wrong..
I configured ImageSharp.Web as described in the (very light) documentation with the very minimal configuration (AddImageSharp and UseImageSharp). I also place a file (a.jpg) in the wwwroot folder of my app. When I access "/a.jpg", the image is displayed correctly, however, when I access "/a.jpg?width=100", I get the following error:
System.ArgumentOutOfRangeException: The UTC time represented when the offset is applied must be between year 0 and 10,000. (Parameter 'offset')
at System.DateTimeOffset.ValidateDate(DateTime dateTime, TimeSpan offset)
at System.DateTimeOffset..ctor(DateTime dateTime)
at System.DateTimeOffset.op_Implicit(DateTime dateTime)
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.SendResponseAsync(ImageContext imageContext, String key, ImageCacheMetadata metadata, IImageCacheResolver cacheResolver)
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.ProcessRequestAsync(HttpContext context, IImageResolver sourceImageResolver, ImageContext imageContext, IDictionary`2 commands)
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context)
Simply configure ImageSharp as described in the documentation.
Repro steps:
Eric’s.jpg
and try to resize itThe image '[url here]' could not be resolved
This can be reproduced any time the url contains ’
, even url encoded. https://unicodelookup.com/#’/1
DEBUG
and RELEASE
modeI'm brand new to the ImageSharp codebase, so apologies if this is user error.
Some requests seem to be getting responses with a 200 status code, but a ContentLength of 0 and an empty body. It seems this is mostly in cases where the same request is hitting the server a few times around the same sort of time.
I've had a go at modifying one of the existing tests to assert the ContentLength is not 0, and it does fail for me, please see below.
AzureBlobStorageCacheServerTests
[Theory]
[InlineData(TestConstants.PhysicalTestImage)]
[InlineData(TestConstants.AzureTestImage)]
public async Task CanProcessMultipleIdenticalQueriesAsync(string url)
{
Task[] tasks = Enumerable.Range(0, 100).Select(i => Task.Run(async () =>
{
var command = i % 2 == 0 ? Command : Command2;
using HttpResponseMessage response = await this.HttpClient.GetAsync(url + command);
Assert.NotNull(response);
Assert.True(response.IsSuccessStatusCode);
// I have added this additional assertion below. If I modify the enumerable range to (0, 2), the test passes
// If I leave the enumerable alone and generate a new width in each iteration, the test eventually passes
// It seems to be only multiple identical requests which have the issue
Assert.True(response.Content.Headers.ContentLength > 0);
})).ToArray();
var all = Task.WhenAll(tasks);
await all;
Assert.True(all.IsCompletedSuccessfully);
}
I have a vague notion that we could be hitting imageContext.SendAsync fewer times than ProcessRequestAsync in the ImageSharpMiddleware, which makes me a little suspicious the issue could lie within some of the static caching in this middleware.
The test above is failing for me in VS2019 16.9.3.
DEBUG
and RELEASE
modeI was reviewing the latest AsyncKeyLock/Doorman implementation and have found a race condition that could cause a caller to be blocked forever waiting to receive the lock, or could potentially cause a reader and writer to execute concurrently. Here are the problematic flows:
How to get permanently stuck waiting for the lock
How to cause a reader and writer to execute concurrently on the same key
There are two problems here:
This is a discussion post around features that I'd like to see long term in a Mvc Core image server/middleware.
Currently, this project is built around a URL based API which has limitations and doesn't cover the gamut of what we need to serve the best image for a given situation:
We need different image formats based on the client Accept
header:
(Note that Safari does not announce support for Jpeg2000, but it is apparently supported)
We need different sizes based on the device pixel ration (DPR
), Viewport-Width
and intended display Width
. These are announced with Client Hints
on supported browsers:
We need to set cache headers appropriately when varying by all of these things.
We also want to simplify the amount of work we need to do to handle images in markup with Tag Helpers.
We may want to add an Accepts-CH
header to the response to indicate that we honour Client Hints
automatically via TagHelperComponents
.
We'd like to automatically handle srcset
attribute generation on img
tags.
We'd like to automatically generate markup for picture
elements to simplify the markup generated.
Longer term, further API's are being discussed such as Network Information - this info could allow us to serve 1x images to higher DPI displays if their connection (downlink
) appears to be poor or their EffectiveConnectionType
is 2g
(for example).
And we also need to handle the source imagery probably not being ideal either (i.e, all the existing options for cropping a certain way, etc.)
I'm just kicking the tin can around at the moment with some ideas in my head. Maybe this is a SixLabors.Web
thing - maybe it's something completely different. I just wanted to open up a discussion at this stage.
All the best! ❤️
DEBUG
and RELEASE
modeWhen using Umbraco 9 on Azure Web Apps using Azure Storage to save media I get the following exception in ImageSharp:
SixLabors.ImageSharp.ImageFormatException: Bad method for ZLIB header: cmf=80
at SixLabors.ImageSharp.Formats.Png.Zlib.ZlibInflateStream.InitializeInflateStream(Boolean isCriticalChunk)
at SixLabors.ImageSharp.Formats.Png.PngDecoderCore.ReadScanlines[TPixel](PngChunk chunk, ImageFrame`1 image, PngMetadata pngMetadata)
at SixLabors.ImageSharp.Formats.Png.PngDecoderCore.Decode[TPixel](BufferedReadStream stream, CancellationToken cancellationToken)
at SixLabors.ImageSharp.Formats.ImageDecoderUtilities.Decode[TPixel](IImageDecoderInternals decoder, Configuration configuration, Stream stream, Func`3 largeImageExceptionFactory)
at SixLabors.ImageSharp.Formats.Png.PngDecoder.Decode[TPixel](Configuration configuration, Stream stream)
at SixLabors.ImageSharp.Image.Decode[TPixel](Stream stream, Configuration config)
at SixLabors.ImageSharp.Image.<>c__DisplayClass133_0`1.<Load>b__0(Stream s)
at SixLabors.ImageSharp.Image.WithSeekableStream[T](Configuration configuration, Stream stream, Func`2 action)
at SixLabors.ImageSharp.Image.Load[TPixel](Configuration configuration, Stream stream, IImageFormat& format)
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.<>c__DisplayClass18_0.<<ProcessRequestAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.<>c__DisplayClass18_0.<<ProcessRequestAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at SixLabors.ImageSharp.Web.Middleware.ConcurrentDictionaryExtensions.GetOrAddAsync[TKey,TValue](ConcurrentDictionary`2 dictionary, TKey key, Func`2 valueFactory)
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.ProcessRequestAsync(HttpContext context, IImageResolver sourceImageResolver, ImageContext imageContext, IDictionary`2 commands)
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
at StackExchange.Profiling.MiniProfilerMiddleware.Invoke(HttpContext context) in C:\projects\dotnet\src\MiniProfiler.AspNetCore\MiniProfilerMiddleware.cs:line 121
at Umbraco.Cms.Web.Common.Middleware.UmbracoRequestMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Umbraco.Cms.Web.Common.Middleware.PreviewAuthenticationMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Umbraco.Cms.Web.Common.Middleware.UmbracoRequestLoggingMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContextOfT`1.ProcessRequestAsync()
This issue seems to only be caused when using Azure/Azure storage. When running on localhost with Azure Storage emulator as media storage the image does load without problems.
Not available as I don't know exactly in what circumstances the issues occur. I do know this images doesn't show up when using query parameters on our development environment running in the configuration mentioned below: https://www.wearetriple.com/media/qfsl0o2c/vodafone-header-image-triple.png
Azure Web App S1 tier
Standard LRS Tier Azure Storage
.Net 6
DEBUG
and RELEASE
modePNG decoding of 3 problematic images (included in the project) directly from an Azure blob stream throws ImageFormatException: Bad method for ZLIB header: cmf=251
. This issue is resolved by reading the entire blob stream, copying it to a MemoryStream
, and returning the MemoryStream
(as seen in CustomAzureBlobStorageImageResolver.cs
).
Download project (ImageSharp.Web.Png.DecodeExceptionRepro.zip), create "img" container in Azure blob storage (by default the project connects to a local Azure storage emulator), upload the 3 images in the Images
folder to blob storage, run the project & request the first image with the following URL: http://localhost:5020/img/1.png?width=512. This should result in the exception. Requesting the other 2 images should result in the exception as well.
Now, stop running the project, open Startup.cs
, modify ConfigureServices
to use CustomAzureBlobStorageImageProvider
(which returns a MemoryStream
instead of an Azure blob stream) instead of AzureBlobStorageImageProvider
. Re-run & re-request the URL. It should work.
The Custom...
classes are based on code in master branch of this repo, with slight modifications marked by // CHANGED
comments.
The .png files are valid according to pngcheck.
Install ImageSharp.Web-master on a Mac. Start Visual Studio with ImageSharp.Web.sln
Building ImageSharp.Web does not produce a target netstandard2.0 as wanted by
corresponding line 7 in *.csproj:
<TargetFrameworks>netstandard1.4;netstandard2.0</TargetFrameworks>
Only target netstandard1.4 is built.
That leads to an error when building ImageSharp.Web.Sample application. This application refers to netstandard2.0 target.
Changing line 7 in *.csproj:
<TargetFrameworks>netstandard2.0</TargetFrameworks>
solves the problem.
(same problem in ImageSharp.Web.Test.csproj
)
System.InvalidOperationException: Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.
at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
at System.Collections.Generic.Dictionary`2.TryGetValue(TKey key, TValue& value)
at SixLabors.ImageSharp.Web.Caching.AsyncKeyLock.GetDoorman(String key) in ImageSharp.Web\Caching\AsyncKeyLock.cs:line 76
at SixLabors.ImageSharp.Web.Caching.AsyncKeyLock.ReaderLockAsync(String key) in ImageSharp.Web\Caching\AsyncKeyLock.cs:line 47
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context) in ImageSharp.Web\Middleware\ImageSharpMiddleware.cs:line 187
Not sure of a better place to post this but here is a basic Azure Blob IImageResolver.
https://gist.github.com/jakenuts/8e05fbd275e060775b6999bdf6d60c22
Thanks!
James
DEBUG
and RELEASE
modeSince merging pull request #73 "Return on no commands", the ImageSharpMiddleware
directly hands off request processing to the next middleware in the pipeline when there aren't any commands found.
This prevents any provider fetching images from a remote source (as the Azure Blob Storage Provider does) from running in a "pass-through" scenario without any transforms. Only images present on the filesystem still work, because the StaticFileMiddleware
handles those requests.
It would be extremely unfortunate to need a second, separate middleware to handle those requests given that it would need to completely duplicate the functionality of the already present provider.
I've seen this pop up a few times intermittently during testing. I'm more than a little concerned that this means our AsyncKeyLock
does not do what it should which is to prevent cache reads while writing.
I'm gonna need help here investigating. Tests explicitly use a single instance of the server and cache and it always works on my machine.
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:11.2043310] CanProcessAndResolveImage(url: "http://localhost/azure/SubFolder/imagesharp-logo.p"...) [FAIL]
X CanProcessAndResolveImage(url: "http://localhost/azure/SubFolder/imagesharp-logo.p"...) [577ms]
Error Message:
System.IO.IOException : The process cannot access the file 'D:\a\ImageSharp.Web\ImageSharp.Web\artifacts\bin\tests\ImageSharp.Web.Tests\Release\netcoreapp3.1\is-cache\b\2\d\5\5\2\9\b\f\e\c\d\b2d5529bfecd.png' because it is being used by another process.
Stack Trace:
at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle)
at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize)
at System.IO.File.Create(String path)
at SixLabors.ImageSharp.Web.Caching.PhysicalFileSystemCache.SetAsync(String key, Stream stream, ImageCacheMetadata metadata) in /_/src/ImageSharp.Web/Caching/PhysicalFileSystemCache.cs:line 128
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.ProcessRequestAsync(HttpContext context, IImageResolver sourceImageResolver, ImageContext imageContext, IDictionary`2 commands) in /_/src/ImageSharp.Web/Middleware/ImageSharpMiddleware.cs:line 306
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.ProcessRequestAsync(HttpContext context, IImageResolver sourceImageResolver, ImageContext imageContext, IDictionary`2 commands) in /_/src/ImageSharp.Web/Middleware/ImageSharpMiddleware.cs:line 318
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context) in /_/src/ImageSharp.Web/Middleware/ImageSharpMiddleware.cs:line 215
at Microsoft.AspNetCore.TestHost.HttpContextBuilder.<>c__DisplayClass23_0.<<SendAsync>g__RunRequestAsync|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.TestHost.ClientHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at SixLabors.ImageSharp.Web.Tests.Processing.PhysicalFileSystemCacheServerTests.CanProcessAndResolveImage(String url) in D:\a\ImageSharp.Web\ImageSharp.Web\tests\ImageSharp.Web.Tests\Processing\PhysicalFileSystemCacheServerTests.cs:line 70
--- End of stack trace from previous location where exception was thrown ---
Hi
The current log levels defined here: LoggerExtensions.cs
Are not defined in the best possible way to allow filtering unneeded log messages: Failures and Success all use Information or Debug level.
Can we tune this so that:
We can make a pull request if you agree with the mentioned levels above.
DEBUG
and RELEASE
modeImageSharp.Web is no longer working in asp.net 3.0 (full release version). the error is:
ArgumentException: The path in 'value' must start with '/'. (Parameter 'value')
Microsoft.AspNetCore.Http.PathString..ctor(string value)
Microsoft.AspNetCore.Hosting.StaticWebAssets.StaticWebAssetsFileProvider.StartsWithBasePath(string subpath, out PathString rest)
Microsoft.AspNetCore.Hosting.StaticWebAssets.StaticWebAssetsFileProvider.GetFileInfo(string subpath)
Microsoft.Extensions.FileProviders.CompositeFileProvider.GetFileInfo(string subpath)
SixLabors.ImageSharp.Web.Caching.PhysicalFileSystemCache.GetAsync(string key)
SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.ProcessRequestAsync(HttpContext context, bool processRequest, IImageResolver sourceImageResolver, ImageContext imageContext, IDictionary<string, string> commands)
SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.MigrationsEndPointMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
the fix for this issue is straightforward, in PhysicalFileSystemCache.cs on line 75 there's a call:
string path = this.ToFilePath(key);
the bug occurs when this path doesn't start with forward slash "/". as you can see from the stack trace of the error, the path eventually gets passed to the Microsoft.AspNetCore.Http.PathString
constructor which expects the path to start with forward slash.
EDIT: i initially reported that this was no longer an issue for PhysicalFileSystemProvider in the full 3.0 release but this isn't the case - it's still an issue. it was only working because of my change to ensure the path started with "/". once i got rid of my changes the issue resurfaced. of course both physical and azure provider both use the same caching mechanism and it's the PhysicalFileSystemCache that's causing the issue.
i'd link you to my project but you'd have to setup so many things to get it running it might be easier to just create a new asp.net 3.0 project and try to resize an image.
DEBUG
and RELEASE
modeAccording to the documentation processing operations should use the order in which they are supplied in the query string, but this is currently not the case. Instead, it loops through all processors in the order they are registered and calls the Process
method on each one (even when none of the commands for that processor are provided):
This probably wasn't noticed before, because the order in which the built-in processors are called doesn't affect the end-result (ResizeWebProcessor
, FormatWebProcessor
, BackgroundColorWebProcessor
and JpegQualityWebProcessor
all affect different aspects of an image). All processors also include a check before mutating the image, so that ensured there were no side-effects by always invoking them.
Umbraco required a custom processor to crop images using coordinates (representing a crop box/rectangle), but adding that after the default built-in processors would result in cropping the already resized image, instead of using the original.
You can use the CropWebProcessor
from umbraco/Umbraco-CMS#10623 to test this issue (or create a similar simple one):
ResizeWebProcessor
, adding the custom processor and then re-adding it again will correctly crop before resizing.The ImageSharpMiddleware
already keeps track of all known processing commands, filters unknown ones from the current request and short-circuits when there are none. This would mean only the above Process
extension method will need to be updated.
Besides invoking the processor in the order they are provided in the query string, we need to ensure they aren't invoked for every command (e.g. twice when using ?width=200&height=200
). This can be done by tracking the all commands used by a processor:
var processedCommands = new List<string>();
foreach (KeyValuePair<string, string> command in commands)
{
if (processedCommands.Contains(command.Key, StringComparer.OrdinalIgnoreCase))
{
// Skip commands already used by a processor
continue;
}
foreach (IImageWebProcessor processor in processors.Where(p => p.Commands.Contains(command.Key, StringComparer.OrdinalIgnoreCase)))
{
source = processor.Process(source, logger, commands, commandParser, culture);
processedCommands.AddRange(processor.Commands);
}
}
Or by tracking available processors and remove them after invoking:
var availableProcessors = processors.ToList();
foreach (KeyValuePair<string, string> command in commands)
{
foreach (IImageWebProcessor processor in availableProcessors.Where(p => p.Commands.Contains(command.Key, StringComparer.OrdinalIgnoreCase)).ToList())
{
source = processor.Process(source, logger, commands, commandParser, culture);
// Prevent invoking the processor again
availableProcessors.Remove(processor);
}
}
I haven't tested the above code, but it should give a good start for a PR/fix and might be something to benchmark/compare against each other.
The current API (using IDictionary<string, string> commands
) combines duplicate query string keys, making it impossible to run processors more than once using different commands, e.g. resize width to 200, crop and then resize the cropped version to a height of 300 (there are definitely better examples to illustrate the use case).
This might be because QueryHelpers.ParseQuery()
is used, which creates an IDictionary<string, StringValues>
and thereby already loses the order of duplicate query string keys. Because the StringValues
are then converted into a single string (joining multiple values with a comma), you might end up with invalid values (e.g. ?width=100&width=200
will result in 100,200
, which gets interpreted as 100200
when used as resize width)!
Also something to note: the cache key only uses known processor commands, so adding cache busting parameters (e.g. rnd
or v
) doesn't currently work. There is a CacheBusterWebProcessor
, but that's only used/added for tests. Changing the order does though, so that part should already work as expected!
Hi I'm trying ImageSharp.Web for the first time so, bear with me.
I created a fresh .Net Core 2.2 project and added the default configuration like you have on your documentation. In this scenario it works like it's supposed to but as soon as you change your static content folder it always returns the original unmodified image.
Scenario 1 (this works).
Use this Configure method (ommited the ConfigureServices method).
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseMvc();
app.UseImageSharp();
app.UseStaticFiles();
}
Scenario 2 (this always returns the original image).
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseMvc();
app.UseImageSharp();
app.UseStaticFiles(new StaticFileOptions
{
FileProvider = new PhysicalFileProvider("c:\\ImageSharpTest"),
RequestPath = "/StaticFiles"
});
}
Checked the folder permissions and I have read and write permissions.
Gut feeling, but it looks like the hard lock is less efficient than what a ConcurrentDictionary would do. I ran https://github.com/benaadams/Ben.BlockingDetector (/cc @benaadams) and it showed up in the logs.
The usage of EntityTagHeaderValue
is like this:
When an application using ImageSharp.Web is using a newer version of Microsoft.Net.Http.Headers
, literally 2.0.0, then this constructor is not available anymore as it has been converted to a StringSegment
.
It results in a MissingMethodException
whenever the middleware is used.
There are two solutions:
Please advise on your preferred solution and I will implement it. At this point I personally don't think supporting 1.4 still makes sense as you released after 2.0.
DEBUG
and RELEASE
modeUnder heavy load the LRUCache may return what seems to be an empty ImageCacheMetadata
(when I say empty, it is not null, but has no apparent values).
This results in a dictionary error when trying to read the format utilities in PhysicalFileSystemCacheResolver
string path = Path.ChangeExtension(
this.metaFileInfo.PhysicalPath,
this.formatUtilities.GetExtensionFromContentType(this.metadata.ContentType));
deanmarcussen@dean ImageSharp.Web % ~/Scripts/bombardier -c 100 -d 5s -k "http://localhost:2646/imagesharp-logo.png?width=300"
Bombarding http://localhost:2646/imagesharp-logo.png?width=300 for 5s using 100 connection(s)
[=============================================================================================================================================================================] 5s
Done!
Statistics Avg Stdev Max
Reqs/sec 3314.89 1101.36 5268.09
Latency 30.18ms 22.94ms 468.19ms
HTTP codes:
1xx - 0, 2xx - 16654, 3xx - 0, 4xx - 0, 5xx - 7
others - 0
Throughput: 18.61MB/s
On this run 7 500 errors, corresponding to this exception
2021-02-26 08:09:33.2911|1|ERROR|Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware|An unhandled exception has occurred while executing the request. System.ArgumentNullException: Value cannot be null. (Parameter 'key')
at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at SixLabors.ImageSharp.Web.Resolvers.PhysicalFileSystemCacheResolver.OpenReadAsync() in /Users/deanmarcussen/Projects/ImageSharp.Web/src/ImageSharp.Web/Caching/PhysicalFileSystemCacheResolver.cs:line 44
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.SendResponseAsync(ImageContext imageContext,
... truncated for brevity.
I can so far trace this back to
// Now resolve the cached image metadata storing the result.
ImageCacheMetadata cachedImageMetadata = await cachedImageResolver.GetMetaDataAsync();
// ImageCacheMetadata cachedImageMetadata = await
// CacheMetadataLru.GetOrAddAsync(
// key,
// _ => cachedImageResolver.GetMetaDataAsync());
where removing the LRU cache resolves the issue (and obviously goes a lot slower!)
Have not had time yet to dig into what's going on with the LRU cache, i.e. where the GetOrAddAsync
might be going wrong.
But from initial look, it might be the issue where GetOrAdd
is not thread safe, and there's a race going on.
This is why Lazy
was used in the other implementation for the Workers
though I've since read there might be better alternatives.
Where did the LRU implementation come from?
Any chance it's been updated since?
Current repro is on macos, core3, latest pull from with the sample project, but also reported on gitter w / OrchardCore in production on IIS Windows Server 2016.
System.NullReferenceException: Object reference not set to an instance of an object.
at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
at SixLabors.ImageSharp.Web.Caching.AsyncKeyLock.GetDoorman(String key) in ImageSharp.Web\Caching\AsyncKeyLock.cs:line 85
at SixLabors.ImageSharp.Web.Caching.AsyncKeyLock.ReaderLockAsync(String key) in ImageSharp.Web\Caching\AsyncKeyLock.cs:line 47
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context) in ImageSharp.Web\Middleware\ImageSharpMiddleware.cs:line 187
DEBUG
and RELEASE
modeIf you request an image with an unsupported image type, an image is created in the cache. This way it is easier to DDOS the system.
-Add ImageSharp.Web to your asp.net core application
-Request an image with an unsupported type
-See in the console that a new image in the cache is created.
I temporarily solved the problem by the following code:
services.AddImageSharp(x =>
{
x.OnParseCommandsAsync = c =>
{
if (c.Commands.Count == 0)
{
return Task.CompletedTask;
}
// It's a good idea to have this to provide very basic security.
// We can safely use the static resize processor properties.
uint width = c.Parser.ParseValue<uint>(
c.Commands.GetValueOrDefault(ResizeWebProcessor.Width),
c.Culture);
uint height = c.Parser.ParseValue<uint>(
c.Commands.GetValueOrDefault(ResizeWebProcessor.Height),
c.Culture);
if (width > 4000 && height > 4000)
{
c.Commands.Remove(ResizeWebProcessor.Width);
c.Commands.Remove(ResizeWebProcessor.Height);
}
//If file extension is not supported, remove command.
{
string extension = c.Commands.GetValueOrDefault(FormatWebProcessor.Format);
SixLabors.ImageSharp.Formats.IImageFormat format = x.Configuration.ImageFormatsManager.FindFormatByFileExtension(extension);
if (format == null)
{
c.Commands.Remove(FormatWebProcessor.Format);
}
}
return Task.CompletedTask;
};
});
Install ImageSharp.Web-master on a Mac. Start Visual Studio with ImageSharp.Web.sln
Open ImageSharp.Web-master/samples/ImageSharp.Web.Sample/Startup.cs
.
Any reference to ImageSharp classes are red-lined.
If you change line 12 of ImageSharp.Web-master/samples/ImageSharp.Web.Sample/Startup.cs/ImageSharp.Web.Sample.csproj
from
<ProjectReference Include="..\..\src\ImageSharp.Web\/ImageSharp.Web.csproj" />
to
<ProjectReference Include="../../src/ImageSharp.Web/ImageSharp.Web.csproj" />
the problem is solved.
DEBUG
and RELEASE
modeThe BackgroundColorWebProcessor doesn't work. It doesn't matter if you try to set a hex or rgb value, you will still get the same result.
/imagesharp-logo.png?width=315&bgcolor=red
instead of Rgba32 converter (that by the way is not registered in the CommandParser) we should use the Color Converter
public FormattedImage Process(FormattedImage image, ILogger logger, IDictionary<string, string> commands)
{
//Rgba32 background = CommandParser.Instance.ParseValue<Rgba32>(commands.GetValueOrDefault(Color));
Color background = CommandParser.Instance.ParseValue<Color>(commands.GetValueOrDefault(Color));
if (background != default)
{
image.Image.Mutate(x => x.BackgroundColor(background));
}
return image;
}
DEBUG
and RELEASE
modeCurrently cached images are loaded into memory unnecessarily: it should be enough to copy the file/azure blob/redis/memory etc. stream into response.Body
without reading the full contents.
IImageCache.GetAsync(...)
, IImageCache.SetAsync(...)
and IImageResolver.ResolveImageAsync()
should work with a dedicated abstract ICachedImage
object instead of an in-memory buffer:
public interface ICachedImage
{
Stream OpenRead();
Stream OpenWrite();
}
For me it seems this API can cover all the use cases which are being solved with in-memory buffers at the moment:
FormattedImage
response.Body
, MemoryStream
)In was able to successfully install ImageSharp.Web in VS 2017 with the command:
install-package SixLabors.ImageSharp.Web -version 1.0.0-beta0005
However, when I tried to add the line:
services.AddImageSharp();
to StartUp.cs, VS 2017 did not recognize the name space - I got an intellisense error and was unable to proceed. This seems to be the same as the problems reported in issue #190 except for the fact that I am using the package manager command line to install.
I tried this in both an existing project and a brand-new one.
Windows 10 VS2017 vanilla - latest version
Install latest Imagesharp.Web beta 0.3
reg service inside startup.cs with:
services.AddImageSharp();
Nothing happens when trying to call ?height=100 on any images from wwwroot.
Download startup template for asp.net boilerplate here: https://aspnetboilerplate.com/
Set full .NET framework.
Add own connectionstring
Add nuget package imagesharp.web
Add services registration
.net core 2.1
full framework 4.6.1
Im still new to Core could anyone explain to me what version i should use when using a combination of core and framework.
Thank you for an excellent library i used imageprocessor.web quite alot, woul be very lovely to get this working with abp's new template.
DEBUG
and RELEASE
modeThere is a couple of reported memory related issues, and while I know there is some tracking in ImageSharp itself about releasing retained resources, there looks to be another issue here (easily resolved) regarding stream usage.
Reported here on Orchard Core OrchardCMS/OrchardCore#9060
and likely related to this discussion #161
Performance testing locally this causes memory usage to grow dramatically - got as high as 15Gb before I stopped it.
Collecting manually doesn't drop it, so I suspect that's why there's reported app service failures.
This line doesn't dispose of the cache file streams
adding a dispose
using (var stream = await cacheResolver.OpenReadAsync())
{
await imageContext.SendAsync(stream, metadata);
}
and everything gets collected when the GC runs, and memory usage is as one would expect.
I've seen some (difficult to reproduce) issues with file locks on the cache images, but I am hoping this might explain those as well.
Off for the covid jab in a minute, so pr to come
Just locally.
DEBUG
and RELEASE
modeCheers for all the fixes over the weekend @JimBobSquarePants .
I'm just doing some integration work with ImageSharp and OrchardCore at the moment and wanted to clarify the behaviour of the ImageMetaData LastModifiedDate, which is used to generate the etag.
The comments all say that it's based on the date the source file was modified
/// <param name="lastWriteTimeUtc">The date and time in coordinated universal time (UTC) since the source file was last modified.</param>
but in ImageSharpMiddleware
when creating the cached image metadata it uses DateTime.UtcNow
, rather than the last modified date of the source image.
cachedImageMetadata = new ImageMetaData(DateTime.UtcNow, image.Format.DefaultMimeType, maxAge);
Unless I've missed something somewhere else?
I can't decide if I prefer it this way, or not, but figured I should clarify if it's intended / unintended?
DEBUG
and RELEASE
modeI implemented ImageSharp.Web to leverage resizing on a website. Some page templates on this site feature the same image, in different sizes, in close succession. I noticed that the image that was resolved first in the group, sometimes became reused for the second request. This lead to the site returning the same image for two distinct query strings. I then dug a bit deeper and managed to reproduce the issue in an automated test. Please have a look at this.
Microsoft.AspNetCore.TestHost
, NUnit
, NUnit3TestAdapter
and SixLabors.ImageSharp.Web
installed.ResizerTests.cs
.images/product.jpg
(has to be wider than 400px).client.GetAsync(...)
produces unique image sizes, Task.Run(() => client.GetAsync(...))
does not.DEBUG
and RELEASE
modeI noticed our application that is hosting the ImageSharp.Web middleware has grown to fill a large percentage of both machines it is on. After analysing a dump file with dotMemory it appears most of the memory is being used by largeArrayPool
in ArrayPoolMemoryManager
I have a feeling this is something I am doing wrong in one of the pieces I have replaced.
It seems to be mostly in a single set of buffers.
The only non standard part of the middleware is preventing caching
public class NoCacheImageCache : IImageCache
{
public Task<CachedInfo> IsExpiredAsync(HttpContext context, string key, DateTime minDateUtc)
{
return Task.FromResult(new CachedInfo(true, DateTimeOffset.UtcNow, 0));
}
public Task<IByteBuffer> GetAsync(string key)
{
return null;
}
public Task<DateTimeOffset> SetAsync(string key, IByteBuffer value)
{
return Task.FromResult(DateTimeOffset.UtcNow);
}
public IDictionary<string, string> Settings { get; set; } = new Dictionary<string, string>();
}
and loading the files from s3
public class S3FileSystemResolver : IImageResolver
{
private readonly IS3Storage _s3Storage;
private readonly Serilog.ILogger _logger;
private readonly IBufferManager _bufferManager;
public S3FileSystemResolver(IS3Storage s3Storage, Serilog.ILogger logger, IBufferManager bufferManager)
{
_s3Storage = s3Storage;
_logger = logger;
_bufferManager = bufferManager;
}
public Task<bool> IsValidRequestAsync(HttpContext context, ILogger logger)
{
var displayUrl = context.Request.Path;
return Task.FromResult(Path.GetExtension(displayUrl).EndsWith(".jpg") ||
Path.GetExtension(displayUrl).EndsWith(".png"));
}
public async Task<IByteBuffer> ResolveImageAsync(HttpContext context, ILogger logger)
{
var bucketName = _s3Storage.BucketNameFor<ImageAsset>();
var displayUrl = context.Request.Path;
_logger.Information("Getting image for {ImageUri}", displayUrl);
var imageId = Path.GetFileNameWithoutExtension(displayUrl);
_logger.Debug("Image id is {ImageId}", imageId);
var imageData = await _s3Storage.DownloadBytes(bucketName, $"{imageId}.png");
if (imageData == null)
{
_logger.Debug("No image found for {ImageId}", imageId);
return null;
}
_logger.Debug("Found image {ImageId}", imageId);
var buffer = _bufferManager.Allocate(imageData.Length);
imageData.CopyTo(buffer.Array, 0);
return buffer;
}
public Func<HttpContext, bool> Match { get; set; } = _ => true;
public IDictionary<string, string> Settings { get; set; } = new Dictionary<string, string>();
}
and setting of the PNG Encoder
public class PngEncoderProcessor : IImageWebProcessor
{
public FormattedImage Process(FormattedImage image, ILogger logger, IDictionary<string, string> commands)
{
var extension = commands.GetValueOrDefault(PngColorTypeCommand);
if (extension.IsNotNullOrWhiteSpace() && Enum.TryParse<PngColorType>(extension, out var pngColorType))
{
image.Image.GetConfiguration().ImageFormatsManager.SetEncoder(ImageFormats.Png,
new PngEncoder { PngColorType = pngColorType });
}
else
{
image.Image.GetConfiguration().ImageFormatsManager.SetEncoder(ImageFormats.Png,
new PngEncoder { PngColorType = PngColorType.Palette });
}
return image;
}
/// <summary>
/// The command constant for format
/// </summary>
private const string PngColorTypeCommand = "pngcolortype";
/// <summary>
/// The reusable collection of commands
/// </summary>
private static readonly IEnumerable<string> PngEncoderCommands
= new[] { PngColorTypeCommand };
public IEnumerable<string> Commands => PngEncoderCommands;
}
services.AddImageSharpCore()
.SetRequestParser<QueryCollectionRequestParser>()
.SetBufferManager<PooledBufferManager>()
.SetCache<NoCacheImageCache>()
.SetCacheHash<CacheHash>()
.SetAsyncKeyLock<AsyncKeyLock>()
.AddResolver<S3FileSystemResolver>()
.AddProcessor<ResizeWebProcessor>()
.AddProcessor<FormatWebProcessor>()
.AddProcessor<BackgroundColorWebProcessor>()
.AddProcessor<PngEncoderProcessor>();
Looks like I cannot inject an EF Core context into an IImageProvider as the lifecycles are not compatible:
System.InvalidOperationException: Cannot consume scoped service 'MyWebApp.Data.ApplicationDbContext' from singleton 'SixLabors.ImageSharp.Web.Providers.IImageProvider'.
I was trying to implement a provider and resolver to display images by id/guid and need to access the database to fetch the actual storage location. But seems it is not possible atm, or am I missing something?
UnauthorizedAccessException when accessing images on web server. Works locally.
UnauthorizedAccessException: Access to the path '~path~\wwwroot\is-cache\2\8\b\3\8\7\f\6\8\b\0\8' is denied.
System.IO.__Error.WinIOError(int errorCode, string maybeFullPath)
System.IO.Directory.InternalCreateDirectory(string fullPath, string path, object dirSecurityObj, bool checkHost)
System.IO.Directory.InternalCreateDirectoryHelper(string path, bool checkHost)
System.IO.Directory.CreateDirectory(string path)
SixLabors.ImageSharp.Web.Caching.PhysicalFileSystemCache.SetAsync(string key, Stream stream, ImageMetaData metadata)
SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
Configuration for startup.cs
class.
ConfigureServices
method:
services.AddImageSharp();
Configure
method:
app.UseImageSharp();
app.UseStaticFiles();
ImageSharp.Web v1.0.0-beta0007
Windows Server 2012 R2
.NET Framework version: 4.7.2
AspNetCore v2.2.0
Hi,
I recently discovered this package and from the looks of it, is exactly what i need for a .net core project. I'm testing it out on a fresh .Net Core 2.0 project and have installed via nuget the package and configured the Startup.cs's ConfigurationServices appropriately, by adding the line services.AddImageSharp();
and have even added inside of the Configure(IApplication app, IHotingEnvironment)
the line app.UseImageSharpe()
Following this i went to test it out by appending a simple crop query string to an image <img src="~/images/banner1.svg?width=100&height=100" alt="ASP.NET" class="img-responsive" />
and nothing seems to be taking any effect, the image remains the same. Is this a matter of further configuration or some issue? any help would be greatly appreciated, thanks!
In the ImageSharpMiddleware
class, the following comment currently exists:
// TODO: How to prevent the allocation in outStream? Passing a pooled buffer won't let stream grow if needed.
This could be achieved by a custom MemoryStream
-esque implementation that grows the buffer by using an array pool or a buffer manager rather than allocating on the heap. (MemoryStream
performs this allocation in its Capacity
property, which makes it a poor candidate for inheriting and changing that specific behaviour, so we'd effectively have to roll our own Stream
to prevent said allocation)
I reckon in high-performance scenarios the difference between having a pooled vs allocated output stream could be significant.
Is this middleware component part of the official release, and would it be worth it to invest the time to optimise this part of the middleware? If so, I could have a stab at it.
DEBUG
and RELEASE
modeThe nightly versions numbers are not generating the expected value. 1.0.0-ci0010
is currently the latest but is lower than 1.0.0-ci0032
published in November.
The ImageSharp repo is working well so we should copy that.
DEBUG
and RELEASE
modeWhen inserting the attached image, it refuses to show and produces the following fail in VS Debug Console;
Microsoft.AspNetCore.Server.Kestrel[13]
Connection id "0HM9FH4P35QNA", Request id "0HM9FH4P35QNA:00000003": An unhandled exception was thrown by the application.
SixLabors.ImageSharp.UnknownImageFormatException: Image cannot be loaded. Available decoders:
- JPEG : JpegDecoder
- PNG : PngDecoder
- BMP : BmpDecoder
- TGA : TgaDecoder
- GIF : GifDecoder
at SixLabors.ImageSharp.Image.Load[TPixel](Configuration configuration, Stream stream, IImageFormat& format)
at SixLabors.ImageSharp.Web.FormattedImage.Load(Configuration configuration, Stream source)
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.<>c__DisplayClass18_0.<<ProcessRequestAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.<>c__DisplayClass18_0.<<ProcessRequestAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at SixLabors.ImageSharp.Web.Middleware.ConcurrentDictionaryExtensions.GetOrAddAsync[TKey,TValue](ConcurrentDictionary`2 dictionary, TKey key, Func`2 valueFactory)
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.ProcessRequestAsync(HttpContext context, IImageResolver sourceImageResolver, ImageContext imageContext, IDictionary`2 commands)
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
Attempt to use the attached image with, in my case, "?height=600&width=600&rmode=max", although other dimensions also have the same issue.
DEBUG
and RELEASE
modeI'm using RC-1 with the azure blob store provider and getting a 404 on any images stored on Azure (local images work fine). I think the issue lines on line 92 of AzureBlobStorageImageProvider.cs.
When extracting the container key from the URL, instead of
string nameToMatch = index != -1 ? path.Substring(index) : path;
should it not be
string nameToMatch = index != -1 ? path.Substring(0 ,index) : path;
to extract the first portion of the path?
I've copied the code for the Azure provider directly into my project and made the above change and it's resolved my issue so i'm reasonably confident in the above.
Attempt to resize an image stored on an Azure blob
DEBUG
and RELEASE
modeThe ImageSharpCoreBuilderExtensions.RemoveProvider<TProvider>
helper extensions don't work.
Call .RemoveProvider
and then check the Service Collection afterwards.
The ServiceCollection is an ICollection
so Remove
needs to find the instance that exists of the ServiceDescriptor
rather than a new instance created by ServiceDescriptor.Singleton<IImageProvider, TProvider>()
Example (from the ServiceCollectionDescriptorExtensions.Replace
method)
var registeredServiceDescriptor = collection.FirstOrDefault(s => s.ServiceType == descriptor.ServiceType);
if (registeredServiceDescriptor != null)
{
collection.Remove(registeredServiceDescriptor);
}
Implementation factories are different again, as it doesn't register the TImplementationType
when creating a ServiceDescriptor
so you need to examine the factory return type
var registeredServiceDescriptor = services.FirstOrDefault(x =>
x.ServiceType == typeof(IImageProvider) &&
x.Lifetime == ServiceLifetime.Singleton &&
x.ImplementationFactory.Method.ReturnType == typeof(TProvider));
if (registeredServiceDescriptor != null)
{
collection.Remove(registeredServiceDescriptor);
}
I have some additional static folders set up
app.UseImageSharp(); // Add the image processing middleware.
app.UseHttpsRedirection(); // wwwroot
app.UseStaticFiles(new StaticFileOptions {
OnPrepareResponse = ctx => {
ctx.Context.Response.Headers.Append(
"Cache-Control", $"public, max-age={cacheMaxAge}");
}
});
// Public folder
app.UseStaticFiles(new StaticFileOptions {
OnPrepareResponse = ctx => {
ctx.Context.Response.Headers.Append(
"Cache-Control", $"public, max-age={cacheMaxAge}");
},
FileProvider = new PhysicalFileProvider(
Path.Combine(Directory.GetCurrentDirectory(), "Public")),
RequestPath = "/Public",
});
The wwwroot folder works fine, but how to support the "Public" folder?
error console:
SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware[2]
The image 'https://localhost:5001/Public/news/67a8376d-6bcc-4d32-b191-7c728a02df67/source.png?width=200&height=50&rsampler=nearest&rmode=stretch' could not be resolved
This issue is designed to capture the architecture and tasks required to build an extensible library for parsing URL API instructions and caching the result.
Feel free to discuss below. We'll assign specific issues for functional components as we determine them.
IImageService
implementations for physical file system and remote images.IDistributedCache
implementation for physical file system that can be switched out for Redis, Azure, etc. How do we handle cleanup, Response codes?The following gives an overall summary of the image processing pipeline.
IImageService
is assigned using URL prefix pattern matching (Regex?)IImageService
determines validity of request and attempts to return stream containing image.IDistributedCache
implementation checks if file exists. (Redirect if exists?)Updated Requirements and Pipeline below
OnBeforeSave
IRequestParser
implementation for parsing requests for command string that can be switched outIImageResolver
implementation collection for physical file system thatCommandParser
implementation for converting commands into real values for processing.IImageWebProcessor
for ResizeIImageWebProcessor
for QualityIImageWebProcessor
for FormatIImageWebProcessor
for BackgroundColorIImageCache
implementation for physical file system that can be switched out for Azure, etc.ILogger
based loggingTagHelper
for generating img
, picture
markupThe following gives an overall summary of the image processing pipeline.
IUriParser
extracts commands; if not found, exitIImageResolver
is assigned using Match()
methodIImageResolver
determines validity of request and attempts to return stream containing image.IImageCache
implementation checks if file exists. (Returns correct response)IImageWebProcessor
implementations process the imagecopied from SixLabors/ImageSharp#216
DEBUG
and RELEASE
modeUsing the AddProvider()
builder extension with an implementation factory throws an exception.
System.ArgumentException
HResult=0x80070057
Message=Implementation type cannot be 'SixLabors.ImageSharp.Web.Providers.IImageProvider' because it is indistinguishable from other services registered for 'SixLabors.ImageSharp.Web.Providers.IImageProvider'.
Parameter name: descriptor
Source=Microsoft.Extensions.DependencyInjection.Abstractions
StackTrace:
at Microsoft.Extensions.DependencyInjection.Extensions.ServiceCollectionDescriptorExtensions.TryAddEnumerable(IServiceCollection services, ServiceDescriptor descriptor)
at SixLabors.ImageSharp.Web.DependencyInjection.ImageSharpCoreBuilderExtensions.AddProvider(IImageSharpCoreBuilder builder, Func`2 implementationFactory) in E:\Sites\ImageSharp.Web\src\ImageSharp.Web\DependencyInjection\ImageSharpCoreBuilderExtensions.cs:line 156
at SixLabors.ImageSharp.Web.Sample.Startup.ConfigureServices(IServiceCollection services) in E:\Sites\ImageSharp.Web\samples\ImageSharp.Web.Sample\Startup.cs:line 22
Add the following code to the samples project Startup
.AddProvider(sp => new PhysicalFileSystemProvider(
sp.GetRequiredService<IOptions<ImageSharpMiddlewareOptions>>(),
sp.GetRequiredService<IHostingEnvironment>(),
sp.GetRequiredService<MemoryAllocator>()))
Clone of ImageSharp.Web
We want to refactor the code so that we produce 3 packages.
SixLabors.ImageSharp.Web.Core
- the core processing engine with its own abstractions around the http request to normalise request formats between core and systemweb (should be netstandard only)SixLabors.ImageSharp.Web.AspNetCore
- the aspnet core middleware piece, this includes the extension methods for configuring in an aspnetcore style + the a middleware dispacher that calls out to the SixLabors.ImageSharp.Web.Core
after translating the request context into an ImageRequestContext
and handling the response.SixLabors.ImageSharp.Web.SystemWeb
- the full framework HttpModule
based implementation like other version responsible for translating between HttpContext
and ImageRequestContext
and calling the processing pipeline.This will allow us to reach older installations targeting full framework and also would provide a transition option for those user using the older ImageProcessor.Web library allowing ImageProcessor.Web and ImageProcessor to be able to be mothballed as users would now have a transition option.
DEBUG
and RELEASE
modeEver since we merged the Lock Free PR #34 we've had intermittent build failures on Travis where tests simply time out. It could be a poorly written test, it could be something more sinister. We should investigate.
/cc @sebastienros
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.