lopezio / wosaic Goto Github PK

A major part of the software development process should be creating tests
and testing during the development process.  We've sort of neglected that
so far, which is partially because it's difficult to do with a GUI
application.  However, I think we should revisit the idea, with some
possible solutions:

-- We could create a complete command-line interface.  This could either be
part of the Wosaic application, or separate.  This would be cool to have
anyway, in my opinion.  Then, we could write command-line tests with
different options
-- Or, we could write command-line test programs that use the core Wosaic
API for its tests.  Each one would be a specific scenario, so it wouldn't
need to be as robust as a full command-line interface.

Obviously, the tests should include boundary cases and whatnot.  Also, we
could use the JUnit testing framework

I was going through the code (to add javadoc comments for public members),
and saw a few public data members that don't neccessarily need to be
public.  These are data members that shouldn't be changed by any other
entity, so we should probably make them private.

The example I first came across was "imagesReceived" in the Controller
class.  Actually, this particular member is initialized, but then never
used by any other source.  It may be a relic of some previous implementation.

I'll post more "encapsulation candidates" as I come across them.

What steps will reproduce the problem?
1. Run the WosaicUI as an applet from a Mac
2. Observe

I expect to see the text fields to have some width, however they seem to only 
be a sliver on the 
actual UI.  This only seems to happen on my Mac, and on the Mac in SC0216.  
Maybe there is some 
default size we have to set.

When creating a mosaic and displaying it inside of the applet, the
individual "Pixels" do not scale with the resolution of the window. 
Instead, each one is a clipped 75 x 75 icon.  The result is that we only
display the top left corner of each pixel, which changes the "average
color" of the Pixel.

Whenever the window is moved, resized, or minimized/maximized, the content
panel loses all of it's content.  This is a result of the underlying JPanel
calling repaint() on itself.

We will eventually fix this by creating a new class that subclasses JPanel,
and then using that instead.

What steps will reproduce the problem?
1. Create a mosaic from Carl's black and white barn image (640x480) or set 
resolution to a very low 
number
2. Create Mosaic
3. Try to save

What is the expected output? What do you see instead?
  - This generates an exception, citing an array index out of bounds exception.

Please use labels and text to provide additional information.

I think it would also be useful to have a sort of creation "wizard", which
steps the user through setting up different parameters.  This doesn't have
to be the only creation process, or even the main one, but I think it will
help beginning users.

We heard somebody mention that using TIFF is much easier and memory
efficient.  We should try it for saving our mosaics.

What steps will reproduce the problem?
1. Launch the application and go to Advanced Options
2. Either enable or disable all of the plugins
3. Resize the Window a little so the panels repaint, and the empty list
will also resize

What is the expected output? What do you see instead?
Even when the lists are empty, they should remain the same size.  This
seems to be an issue with either the JList component, or the containing
JScrollPane, or the way it's laid-out on the screen altogether-- it's hard
to say.

What steps will reproduce the problem?
1. Set the number of images to use equal to a number less than 10
2. Try genereating a mosaic

What is the expected output? What do you see instead?

The  mosaic should be generated with this number of Flickr images, however the 
Flickr thread 
returns right away.  This is probably due to some rounding on integer division 
somewhere.

What steps will reproduce the problem?
1. Pick any image as a source
2. Tune the resolution to, say, 500
3. Hit 'Generate Mosaic' and watch it chug along confusedly

The goal is to quickly update the UI so that the user can see progress.  
Currently we are signaling an 
event every time the mosaic is updated.  For large resoltions, this may be too 
often.  Additionally, it 
is probably not wise to do all the work of updating in the AWT Event Thread 
itself.  Instead, our 
listeners that catch these events should probably add to a queue that is 
running in a separate 
thread (pool?).

This is a minor issue that we discussed before, but I think we can addresss
it now when we're doing our re-write.

If we pass a JDialog from each of the source plugins, then they can define
many more options (such as the size of the window, etc.), and we have
better control of whether we want the dialog to me "modal" or not.

I've started implementing this in the rewrite branch, but it still needs a
little more work.

Right now, the algorithm for finding pictures in the FilesystemPlugin uses
a recursive algorithm.  This is probably not very memory efficient, and
might even introduce infinite loops with symbol links.  It would be easy to
change it to add recursive directories to a processing queue, and then
spawn queries off of that.

Facebook might be another potential candidate for adding features.  Right
now we just get pictures of a logged-in user.  But, we could get pictures
in the user's album, a friend's album, or just pictures of a friend. 
There's all sorts of potential combinations we haven't explored yet.

In the main interface, I think it would be easy and beneficial to the user
if we had tooltips display on mouseover for many of the components.  This
should supplement some other "Help" scheme we come up with in the future

Currently, plugins (as well as others) report their progress through a
direct reference to the "Status" object in the WosaicUI.  This is bad
because it breaks encapsulation, and is a clear case where we could use
actions and actionlisteners for the same functionality.  The added benefits
are:

-- Better encapsulation, mainly
-- The ability to add multiple status "listeners", or none.  Alternatives
could include:
   -- Simply stdout or stderr
   -- A log file
   -- More prominent alerts for high-priority updates, such as dialogs

This could be useful if we want to create, say, a command-line interface or
a simple test suite.

JAI Has been bad news for this project from the start.  It is complicated and 
full of memory holes.  
There are trace remnants of JAI (such as internal representation of images as 
RenderedOps).  To 
save memory and be consistent, we should get rid of all JAI references.

Right now, we just show a generic progress bar scrolling while we generate
an image.  Previously, we used a fixed-number of images that we planned on
retrieving to report progress.  Neither of these is optimal.  We should
have some sort of call to each plugin to get our "expected" number of
returned images, and show progress based on that.

We can improve our mosaics by improving our algorithm for matching images.
 Moreover, the user should be able to pick between many algorithms for
options.  We should first kind of "abstract" the process of creating and
processing color data in a pixel object.  Then we can dynamically change
the algorithm.

Possible additions the algorithm include:
-- Lessen the number of duplicate images that get used
-- Break each pixel region into sub-regions, and score based on the
matching in each sub-region.
-- Edge detection?

Some of the wiki pages are out of date.  Listed below are the pages we need to 
update, and how we 
should update them.


   * FuturePlans - we have implemented parallel threads of queries, and custom sources.
   *  ControlFlow - the diagram and description need to be updated to take custom sources into 
account
   * README - this will be replaced with the series of documentations provided with the source code
   * Progress - this page might be better served as an 'About' page

Proposed additions to the wiki are also a document describing the interface for 
the SourcesPlugin.

Problem:  Currently there are some circular references between the Controller 
class, FlickrServices, 
and JAIProcessor.  Best practices dictate that we should try to make these 
components as 
independent as possible and provide clean interfaces for interaction.

Specifically, FlickrServices and JAIProcessor use a references to Controller to 
control their thread 
state, and access a shared buffer.

Proposed Solution: Create a shared buffer object that the Controller can 
instantiate and pass to each 
component.

This is more of a planned feature than an issue.  We would like to load the 
source image that the 
user chooses to create a mosaic of into the display area of the application.  
This way the user can 
see what picture they have chosen.

One of the biggest bottlenecks for our program right now is fetching images
from Flickr.  It would be very useful if we could cache at least the image
representations, for use in a next session.  However, we should be careful,
because our cache database could easily blow up, with the number of images
we're fetching per query, and the number of queries we use.

I'm listing this issue just to have as a reminder.  There is a lot we can do to 
the source code of 
Wosaic in order to make it more readable.  I've listed a couple points below:

  * UI Code can be separated into more functions and cleaned up
  * ActionListeners and Dialogue classes can be moved into a different file (maybe)
  * We can use the parameters object, or create a new object to pass around mosaic and source 
plugin parameters

This is more of a reminder to myself.  I was looking through the Mosaic
class, and I think there's a possibility that we can clean up at least a
few different member variables, and the createImage method.

This doesn't have to come before the next release, but I'd like to take a
look eventually

It's kind of a mess-- we hardly use wosaic.ui, and wosaic.util is used for
everything.  We could organize a little

What steps will reproduce the problem?
1. Make a mosaic with resolution >= 75, and source image with resolution close 
to 1000x2000 
pixels
2. Let it finish
3. Make another mosaic with same or greater resolution

What is the expected output? What do you see instead?
A whole slue of exceptions are thrown, and at the base is an "out-of-memory" 
exception.

Please use labels and text to provide additional information.

Sourceforge is a very large site for open source projects, and has quite a
user base.  Launchpad is another, but is generally geared towards Linux
projects.  We should see if there is an easy way to have our hosting on one
of these sites, but mirror our data on the others as well.

When we resize pixels, we generally use a "fast smoothing" algorithm that
doesn't return very good quality.  This doesn't matter for our display,
because they are shown very small.  However, it might be useful to have a
save or preview function that uses higher-quality images.  This would
either use:

-- A higher-precision resizing algorithm on our original fetched images, or
-- Fetch higher-quality images.  For this, we would need to start a
"high-quality" url/file string in each pixel.  Also, we need to be careful
about what sort of caching we do, to avoid large memory consumption.

Users are more used to seeing a menu bar for an application, as well as
toolbars.  If we can put most or all of our components into these, it would
lessen the learning-curve for the application.  At the very least, I think
we should have a menu bar.

I've only encountered the error so far with 1 PNG, so it's hard to say what
the real cause is.

Steps to reproduce:
1. Download attached PNG
2. Start Wosaic and set the source image as wosaic-open.png
3. Add some search string, and hit "Generate"

What steps will reproduce the problem?
1. Choose a search term that does not yield many results
2. Continue generating a mosaic as normal
3. Watch the JAIProcessor never complete

What is the expected output? What do you see instead?

The JAIProcessor never finishes if we don't receive the expected number of 
images in the 
sourcesBuffer.

As a temporary fix, I've added a simple state machine to the ImageBuffer.  It 
will now keep track 
of the number of sources that feed the buffer, and will have a method to signal 
when a source 
completes.  When all n sources have signaled completion, it will set the 
isComplete flag to true, 
regardless of how many items actually entered the queue.

One of the next things we should do is add functionality for zooming in/out on 
the generated 
mosaic (keeping the window size constant but essentially increasing the size of 
each Pixel).  
Additionally, we can try adding an effect that displays an image in a larger, 
preview pane when you 
hover over it with the mouse.

The Flickr API has many more options for searching than we are currently
using.  We should expand our plugin to take advantage of essentially all of
these.  It would include:
-- Advanced string searching
-- Only return creative commons pictures
-- Search for a particular user's pictures
-- Search for a particular group's pictures

For saving, and mosaic generation, we should have some status indicator (ala 
blue progress bar, 
maybe status messages emitted at certain key points in execution) to show the 
user that the 
application is doing work, and to indicate when it has finished.

In our JAIProcessor, we only have one thread that is in charge of
sequentially analyzing a Pixel object and inserting it into the Mosaic. 
With careful synchronization on our Mosaic, we could easily make worker
threads in the JAIProcessor that would speed up this process.

There are quite a few methods and data members that don't have Javadoc
comments associated with them.  To be useful, we need to create Javadoc
comments for each of them.  Public members should be a first priority, but
I think it would be good to have good comments for all of them

Picasa is another great source of pictures that we could exploit.  Google
generally puts out pretty nice APIs-- we should see if we can use it.

There are some options that a user will configure in one session, that
would be useful to persist into the next session.  It would be good to save
these settings into a file, and check for it upon the next run.  Setting
might include:
-- Mosaic Resolution
-- Enabled Plugins
-- Facebook Authentication

There should be easy ways to save this data in either plain text or XML format.

The Swing components we are using look very bad on some platforms
(particularly Linux with GTK).  The SWT kit uses all native components, so
it should look nice on all platforms.  As a consequence, the API is very
different, and the component set is a little more limited.  This would take
a bit more work, but would be quite an improvement.

Using the native call Image.getScaledInstance() for very large images is
very slow.  If we can come up with a similar algorithm using an
AffineTransform, we could potentially speed up the plugin a lot, and make
it much more useful.

The first time I tried to write the code, I ran into problems with images
with transparency.  However, with a little more work, I think we could have it.