enlightenedperlorganisation / task-kensho Goto Github PK

View Code? Open in Web Editor NEW

71.0 71.0 23.0 525 KB

A Glimpse at an Enlightned Perl Distribution

Home Page: http://search.cpan.org/dist/Task-Kensho

License: Other

Perl 98.81% Shell 1.19%

task-kensho's People

Contributors

Stargazers

Watchers

task-kensho's Issues

Add parallelization modules to Async category

Possible candidates: MCE, Parallel::ForkManager, IO::Async::Function (though redundant with IO::Async already there)

Ref: #10 #56

Document "acceptable replacements"?

The documentation for Task::Kensho states, "Each of these modules has a perfectly acceptable replacement that may work better for you." I (for one) would like to see a reference to the list of possible replacements, especially since (from attempting to install it) if memory serves there are some modules (either prerequisites or included as part of Task::Kensho) which do not install on some platforms (for example, Starman, a prerequisite for Pinto [part of Task::Kensho::Toolchain], states in its documentation it does not support Win32).

(I tried to post this as a response to Grinnz's post on blogs.perl.org ("Task::Kensho needs your help!"), but it kept saying my session had expired.)

Add Nice::Try to Exceptions section

All the Perl try/catch implementations suffer from significant deficiencies compared to an ideal syntax. I think Nice::Try comes closest to perfection. No extraneous semi-colons, it has a finally block, but best of all, it supports multiple catch blocks with exception filtering.

Support XLSX

The suggested modules for Excel/CSV are

The first two depend on Spreadsheet::ParseExcel and Spreadsheet::WriteExcel which is fine for .xls files, but neither works on modern .xlsx files.

I suggest to at lease at Spreadsheet::ParseXLSX and Excel::Writer::XLSX to support (more) modern Excel.

For Spreadsheet reader/parser support, I'll be bold to suggest my own Spreadsheet::Read, which is a uniform layer over (most) spreadsheet reader/parser modules (ODS, SXC, XLS, XLSX, CSV, SQ)

I'm not aware of a likewise module for writing (yet).

add HTTP::BrowserDetect to WebDev

this is an excellent module, best-in-class for parsing user agents from web clients.

Add to the docs a list of categories only

Hey,

It would be nice if the docs had a section listing all the available Task::Kensho::* categories. Essentially a list like this, but with the one-two word descriptions:

Last night I was browsing the module, looking for anything recommended for RESTful communications, and it was tough figuring out what sections Kensho even covers. I ended up scrolling through a giant TOC:

I think a list of sections in the docs would be beneficial.

Add Log::Any to Logging category

One of the most widely used and flexible logging frameworks.

Add Mojo::DOM to XML category

Mojo::DOM is also a great way to work with XML documents.

Is this project still maintained?

Last version on cpan was May 2016. I was going to point people to Task::Kensho, but I find its contents not very up-to-date...

add Path::Tiny

for Excel add Excel::Writer::XLXS and Spreadsheet::ParseXLXS

or, Spreadsheet::Read and Spreadsheet::Write

enlightenedperl.org references

The POD for the various sections links to http://www.enlightenedperl.org/ but that URL isn't responding - and is unlikely to again with the EPO being wound up.

These links ought to be removed so as not to confuse users and hopefully to dissuade squatters from grabbing the domain when it lapses.

Prune / Clarify existing Date Modules

Remove Perl::Version

Perl::Version does not actually deal with Perl-style versions, it only correctly works with version tuples which is one of the two Perl version styles (and the less prevalent one at that). See: https://rt.cpan.org/Ticket/Display.html?id=110074, https://rt.cpan.org/Ticket/Display.html?id=114541, https://rt.cpan.org/Ticket/Display.html?id=119447, https://rt.cpan.org/Ticket/Display.html?id=124070, https://rt.cpan.org/Ticket/Display.html?id=54481, https://rt.cpan.org/Ticket/Display.html?id=47501

Modern authors should not need any module to deal with Perl versions of either format since version.pm is used automatically. Version::Next can be used for correct version bumping.

Add Mojo::IOLoop to Async category

Mojo::IOLoop's usage as a general async framework/ecosystem is sufficient at this point to warrant its inclusion in this category.

replace Test::NoWarnings with Test::Warnings

Test::Warnings was written to be a drop-in replacement for Test::NoWarnings that also allows one to use done_testing without running into plan issues; it also covers the interfaces provided by Test::Warn (which Test::Most dragged in).

I'm mostly indifferent as to whether Test::FailWarnings is also included (as in PR#3).

(I vote yes on the first paragraph.)

Add Capture::Tiny - suggestion

Please up/down/side comment

https://metacpan.org/pod/Capture::Tiny

Templating category?

There are enough well-built templating modules that it may warrant its own category. Template can of course be moved there, and possible candidates: Text::Xslate, Text::Template, Mojo::Template

Add Test2::Suite to Testing category

add Time::Moment to Dates section

(I vote yes)

Replace Devel::Repl with Reply

Received via Email:

Hi Chris,

I see that Task::Kensho recommends using Devel::Repl. I had some trouble installing this tool and
did some searching online and I found that the author of that tool appears to recommend a different
repl called Reply. See for details: https://shadow.cat/blog/matt-s-trout/mstpan-17/

That being the case, perhaps an update to Task::Kensho is in order?

Thank you.

Regards,
Rich Duzenbury

Request: add MCE

From: Mario Roy [email protected]
Subject: Searching the string "parallel" at Task-Kensho produce no results

Greetings Karen,

Thank you for the list of modules at https://metacpan.org/pod/Task::Kensho.
The list is very informative.

I searched the page for "parallel" and nothing came up. May I suggest MCE
as a possible candidate. MCE was written to tackle parallelism on a box.
MCE *does not* compete with other modules, but instead complements in the
wild. By itself, MCE provides out of the box parallelism for map and grep.

use MCE::Map;
my @a = mce_map { "Please add MCE $_\n" } 0 .. 1_000_000;

use MCE::Grep;
my @b = mce_grep { $_ % 7 == 0 } 0 .. 100;

One line enables parallelism in Test-Perl-Critic.
https://github.com/Perl-Critic/Test-Perl-Critic/blob/master/lib/Test/Perl/Critic.pm#L104

David Farrell did a nice write up on MCE some time back.
http://perltricks.com/article/61/2014/1/21/Make-your-code-run-faster-with-Perl-s-secret-turbo-module

MCE is a chunking engine. It eats big files for breakfast :). Well,
chunking has the effect of reducing IPC behind the scene.

http://api.metacpan.org/source/MARIOROY/MCE-1.521/images/02_Bank_Model_Chunking.gif

Every chunk comes with an ID. So easy to have ordered output while running.
http://api.metacpan.org/source/MARIOROY/MCE-1.521/images/03_Bank_Model_Chunk_ID.gif

MCE allows natural cross-over function calling. Serialization is done
automatically.
http://api.metacpan.org/source/MARIOROY/MCE-1.521/images/08_Natural_Callback.gif

There is even bin/mce_grep to enable scientists to parse massive genome
files using the binary agrep or tre-agrep commands.
https://metacpan.org/source/MARIOROY/MCE-1.521/bin/mce_grep

The presentation by Gavin Carr takes MCE for a spin.
http://www.openfusion.net/talks/pwp/
http://www.openfusion.net/talks/pwp/#22   mce_grep
http://www.openfusion.net/talks/pwp/#20   mce_loop

For folks who like prime numbers, compare the 2 examples. MCE runs with
Perl not compiled with threads support.

https://metacpan.org/source/DANAJ/Math-Prime-Util-0.49/examples/fibprime-threads.pl
https://metacpan.org/source/DANAJ/Math-Prime-Util-0.49/examples/fibprime-mce.pl

MCE supports an input iterator (complement to Jeffrey Thalhammer). Notice
how the MCE code is much smaller. To use threads (not fork, one simply adds
use threads; at the top).

Well, am hoping to see MCE listed for parallelism. Please... Thank you.

Kind regards,
Mario

----

Hi again,

In other words, may MCE be added to the list of modules at
https://metacpan.org/pod/Task::Kensho for enabling parallelism? I
understand that this request may be rejected. Oh gosh, will feel bad if
that were to happen. Will feel like all the hours was worth it if it were
to be added.

It was no small feat to have MCE run on all environments including
Windows/Cygwin. I am embarrassed to say, but that was hard.

Scientists out in QATAR (are using MCE -- mainly benefiting from workers
persisting between row data.
http://qatar-weill.cornell.edu

Of all things, MCE is helping with parallelism hacking molecules.
https://metacpan.org/source/DEMIAN/HackaMol-0.00_21/examples/MCE/heavy_mvr.pl#L11

I even received a report from Greece stating MCE is used in a Windows
production environment only possible with MCE 1.516 and later releases.

Well, MCE is my humble 2 cents in helping folks enable parallelism in Perl.

Thank you.
Mario

----

Just realized that I need to monitor the perlmonks site and help out answer questions.

Splitting large array for threads.
http://www.perlmonks.org/?node_id=1089915

Look at MCE go, better yet enabling chunking, and furthermore direct processing of the input file.
http://www.perlmonks.org/?node_id=1110247

Several folks have asked for Ruby-MCE, Python-MCE, even Go-MCE. Perl is first to receive MCE.

Work done thus far in Ruby-MCE has helped me solved a rare bug in Perl-MCE on Windows. The bug existed for the longest time and spent countless hours on many occasions trying to reproduce it. The bug presented itself in Greece and finally able to reproduce it.

If I had the time, would love to re-write MCE. Add more features. The next item for Perl-MCE is mce_start sub { ... } based on MCE::Flow and to run in the background. Several mce_start may run, each with various workers, each massively parallel with full chunking support. Also have mce_await, mce_status, etc. I can do this. Will be awesome.

I'm not totally sure about Perl6 and whether it will have chunking abilities. So may need to write Perl6-MCE one day.

Thank you in advance for considering MCE as a parallel module for the list. I would do a pull request, but not sure under what section to place MCE under. Thus the reason for not sending. Will wait for a go before sending a pull request otherwise.

Do I add it after Module Development?

MCE: Parallel Processing

Many-Core Engine for Perl providing parallel processing capabilities

Cheer,
Mario

----

Hi all,

I am ok if MCE is not listed on https://metacpan.org/pod/Task::Kensho. I
have found peace and no longer think about it.

There was one thing remaining with MCE. That is, a massive update on the
documentation itself. MCE 1.522 was released today.

1.522 Thu Dec 25 16:00:00 EST 2014

[BUG FIXES]

* Applied fix to MCE->shutdown so that MCE Models do not err when
  receiving signal to terminate.

* Optimization for MCE->print, MCE->printf, MCE->say and MCE->sendto.
  This was calling fileno unnecessarily. MCE->print('STDERR', ...)
  is not supported. Do this instead; MCE->print(\*STDERR, ...)

[ENHANCEMENTS]

* Inserts the actual lib-path at the head of @INC in example files.
* Massive documentation updates throughout the entire distribution.
* Renamed barrier_sync.pl to sync.pl; scaling_pings.pl to ping.pl.
* Updated requires in META.yml, Makefile.PL and perl-MCE.spec.
* Updated comment in MCE::Queue test scripts.
* Updated the README file.

[NEW FEATURES]

* Added sampledb examples demonstrating DBI and SQLite with MCE.
* Added step_demo.pl to examples.


All 6 models are powerful in MCE. Look at MCE::Grep go below.
https://metacpan.org/pod/distribution/MCE/lib/MCE.pod#MCE-MODELS

A common scenario is grepping for pattern(s) inside a massive log file.
Notice how parallelism increases as complexity increases for the pattern.
Testing was done against a 300 MB file containing 250k lines.

use MCE::Grep <https://metacpan.org/pod/MCE::Grep>;

my @m; open my $LOG, "<", "/path/to/log/file" or die "$!\n";

@m = grep { /pattern/ } <$LOG>;                      ##  0.756 secs
@m = grep { /foobar|[2357][1468][9]/ } <$LOG>;       ## 24.681 secs

## Parallelism with mce_grep. This involves the manager process
## due to processing a file handle.

@m = mce_grep { /pattern/ } <$LOG>;                  ##  0.997 secs
@m = mce_grep { /foobar|[2357][1468][9]/ } <$LOG>;   ##  7.439 secs

## Even faster with mce_grep_f. Workers access the file directly
## with zero interaction from the manager process.

my $LOG = "/path/to/file";
@m = mce_grep_f { /pattern/ } $LOG;                  ##  0.112 secs
@m = mce_grep_f { /foobar|[2357][1468][9]/ } $LOG;   ##  6.840 secs


Merry Christmas !!!

Kind regards,
Mario

Abstracted 'Correct naming' layer

There are many tickets here that basically say ... well some modules are in the wrong place on CPAN, likely at the time it was the only things they partially matched.

However it is quite simple (by modifying an index on a cpan mirror) to get any name to download any perl module.

It would be more than possible to maintain a darkpan that designates its own layout, this is pretty weird in the fact that if you for instance pulled Async::POE(Our designation) you may be fetching from 'POE' (as POE is directly in / normally) but the index its self would serve as an example of how things should be laid out.

More of an exercise in tracking over anything else, but it would give us a wonderfully catagorized index of the things we feel are worthy

I have plenty of spare server resources if a few people fancied giving it a spin :)

IPC category

An IPC category would be useful for recommendations on modules to handle running and managing processes. IPC::Run, IPC::Run3, IPC::System::Simple, and maybe even IO::Async::Process or POE::Wheel::Run.

remove Test::Most

It is mostly just a bundle of other Test::* modules, but also tinkers with test guts in naughty ways (e.g. see https://rt.cpan.org/Ticket/Display.html?id=100213). We are better off recommending individual test modules, and CPAN distributions benefit from having fewer dependencies.

(I vote yes)

Add Future to Async category

Future is a great eventloop-independent tool for async programming.

Dancer should be Dancer2

I think Dancer2 is ready for prime time. What do you think?

Add Excel::Writer::XLSX to Excel/CSV category

Excel::Writer::XLSX

Add Mojo::UserAgent to Web Crawling

Filesystem category

A new filesystem category could include Path::Tiny as suggested in #47, as well as modules like: File::Temp, File::Spec, File::pushd, File::Slurper. IO::All could also be moved here.

Data::Edit::Xml

Please consider adding: https://metacpan.org/pod/Data::Edit::Xml to: https://metacpan.org/pod/Task::Kensho under: Task::Kensho::XML: XML Development ?

Serialization category

To go with #22, a Serialization category could be added (for data transmission, not debugging) where Sereal would easily fit, JSON::MaybeXS could also be moved there. Some other possibilities for consideration: YAML::XS, CBOR::XS.

add Win32 and Crypt categories

perl is well known as a cross-platform tool, but actually on Windows, you have to choose some WinOnly modules, like Win32::OLE etc. So I think a Win32 category is necessory.
Crypt is also a important category in modern CS, please considering to add this too.

Split ModuleDev category?

Right now the Module Development category seems to encompass both "tools to use while developing modules" and "tools for releasing CPAN modules". Perhaps Dist::Zilla, Module::Build::Tiny, and CPAN::Uploader could be moved to a new CPAN dev category?

Wishlist: 'Install All' Option

I'm installing a module set on a clean perl install, and I'm finding the --interactive option is about to make me start foaming from the mouth.

I understand the idea was that it'd ask whether to install particular Kensho sub-tasks, which it did, but it's also asking whether to install each module of each sub-task, _AND_ each optional module of each of those modules.

I think I had this install open for 4 hours now, always coming back to see "Install the X optional module(s)? [y]" in the terminal just sitting there.

This issue will only get more severe, as more modules are added to Task::Kensho.

I use the majority of sub-tasks and the majority of their modules. I don't mind having a few extra interesting modules installed; to play around with to try them out.

A simple "Install all automatically?" question in Task::Kensho itself or even just the sub-tasks would have avoided what is now proving to be an extremely annoying and time-consuming experience for me.

Cheers,
ZZ

Task::Kensho::XML requires Perl 5.14 due to RDF::Trine dependency on IRI

Several other subtasks require 5.10 for various dependencies but this is the only subtask that ends up requiring higher than that.

add Sereal

I'm not sure what category it best belongs in. We don't really have a "low-level systemy tools" section.

This was suggested on irc by @ranguard.

(I vote yes.)

Add ojo to Script Hackery category

There are few better tools for oneliners.

Add Email::Stuffer to Email category

Email::Stuffer is a very convenient wrapper around Email::Sender and Email::MIME, and the first thing I suggest to anyone for sending email in Perl.

Add Getopt::Lucid to a new section (command line processing)

I think the current CLI section holds other kind of modules. There are many candidates, the one I suggest is Getopt::Lucid - Clear, readable syntax for command line processing

MooseX::Getopt could be moved to this new section (from Task::Kensho::OOP)

Recommend Bread::Board?

Don't think Task::Kensho recommends an IoC module at all? I get the impression B::B is pretty solid now, and even has docs these days :-)

Add Machine Learning Category

I know you may be thinking: "first let's add machine learning to perl". But someone ported Apache MXNet not too long ago, and I feel like noone knows about this.

add Web::Simple to WebDev

(I vote yes)

remove TryCatch

It uses Devel::Declare and some other intrusive low-level things, and Try::Tiny has proven itself now.

(I vote yes)

Errors in perl debugger

Hii everyone! :)

I installed Perl and Task::Tensho

But recently, I've started to run into an error in any interactive command
I use. Could you please have a look at it? I really like Perl and I need to
have the interactive commands working.

I use macOS Yosemite

The error log is the following

perl -de 42

Loading DB routines from perl5db.pl version 1.49_04
Editor support available.

Enter h or 'h h' for help, or 'man perldebug' for more help.

main::(-e:1):    42
Cannot do `initialize' in Term::ReadLine::Gnu at
/usr/local/Cellar/perl/5.24.0_1/lib/perl5/5.24.0/perl5db.pl line 6861.
 at /usr/local/Cellar/perl/5.24.0_1/lib/perl5/site_perl/5.24.0/
darwin-thread-multi-2level/Term/ReadLine/Gnu.pm line 844.
    Term::ReadLine::Gnu::AU::AUTOLOAD(Term::ReadLine=HASH(0x7f9363267290))
called at /usr/local/Cellar/perl/5.24.0_1/lib/perl5/site_perl/5.24.0/
darwin-thread-multi-2level/Term/ReadLine/Gnu.pm line 287
    Term::ReadLine::Gnu::new("Term::ReadLine", "perldb",
GLOB(0x7f936218ad68), GLOB(0x7f936216fa78)) called at
/usr/local/Cellar/perl/5.24.0_1/lib/perl5/5.24.0/perl5db.pl line 6861
    DB::setterm() called at /usr/local/Cellar/perl/5.24.0_
1/lib/perl5/5.24.0/perl5db.pl line 1845
    DB::_DB__read_next_cmd(undef) called at /usr/local/Cellar/perl/5.24.0_
1/lib/perl5/5.24.0/perl5db.pl line 2779
    DB::DB called at -e line 1
Cannot do `initialize' in Term::ReadLine::Gnu at
/usr/local/Cellar/perl/5.24.0_1/lib/perl5/5.24.0/perl5db.pl line 6861.
END failed--call queue aborted.
 at -e line 0.

perl -V

Summary of my perl5 (revision 5 version 24 subversion 0) configuration:

  Platform:
    osname=darwin, osvers=14.5.0, archname=darwin-thread-multi-2level
    uname='darwin yosemitevm.local 14.5.0 darwin kernel version 14.5.0: thu
apr 21 20:40:54 pdt 2016; root:xnu-2782.50.3~1release_x86_64 x86_64 '
    config_args='-des -Dprefix=/usr/local/Cellar/perl/5.24.0_1
-Dprivlib=/usr/local/Cellar/perl/5.24.0_1/lib/perl5/5.24.0
-Dsitelib=/usr/local/Cellar/perl/5.24.0_1/lib/perl5/site_perl/5.24.0
-Dotherlibdirs=/usr/local/lib/perl5/site_perl/5.24.0
-Dperlpath=/usr/local/opt/perl/bin/perl
-Dstartperl=#!/usr/local/opt/perl/bin/perl
-Dman1dir=/usr/local/Cellar/perl/5.24.0_1/share/man/man1
-Dman3dir=/usr/local/Cellar/perl/5.24.0_1/share/man/man3 -Duseshrplib
-Duselargefiles -Dusethreads'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.10
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include
-DPERL_USE_SAFE_PUTENV',
    optimize='-O3',
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.10
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion='', gccversion='4.2.1 Compatible Apple LLVM 7.0.2
(clang-700.1.81)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678,
doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16,
longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -mmacosx-version-min=10.10 -fstack-protector-strong
-L/usr/local/lib'
    libpth=/usr/local/lib /usr/include/libxml2 /Applications/Xcode.app/
Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/7.0.2/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/
XcodeDefault.xctoolchain/usr/lib /Applications/Xcode.app/
Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/lib
/usr/lib
    libs=-lpthread -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=, so=dylib, useshrplib=true, libperl=libperl.dylib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags=' -mmacosx-version-min=10.10 -bundle
-undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl):
  Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
                        PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
                        PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
                        PERL_PRESERVE_IVUV PERL_USE_SAFE_PUTENV
                        USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS
                        USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE
                        USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME
                        USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API
  Built under darwin
  Compiled at Jun  4 2016 11:01:07
  @INC:
    /usr/local/Cellar/perl/5.24.0_1/lib/perl5/site_perl/5.24.0/
darwin-thread-multi-2level
    /usr/local/Cellar/perl/5.24.0_1/lib/perl5/site_perl/5.24.0
    /usr/local/Cellar/perl/5.24.0_1/lib/perl5/5.24.0/darwin-
thread-multi-2level
    /usr/local/Cellar/perl/5.24.0_1/lib/perl5/5.24.0
    /usr/local/lib/perl5/site_perl/5.24.0

echo $TERM

xterm-256color

ZSH shell

enlightenedperlorganisation / task-kensho Goto Github PK

task-kensho's People

Contributors

Stargazers

Watchers

Forkers

task-kensho's Issues

Recommend Projects

Recommend Topics

Recommend Org