go-python / gpython Goto Github PK

During I add a test for #40.
I found that this syntax does not work as I expected.

a = range(100, 0, -1)
len(a) <- 100
b = [a for e in a]
len(b) <- should be 100 but 0 in gpython

The ReadMe.md says:

`
It does not include very many python modules as many of the core modules are written in C not python. The converted modules are:

builtins
marshal
math
time
sys
`

What you need is the Ourobours library.

A standalone, pure Python implementation of the Python Standard Library. This allows Python default libraries to be used portably on any implementation of the Python virtual machine---not requiring CPython.

And here is the list of Modules.
https://github.com/beeware/ouroboros

Files 1488
Lines Of Code 419872

That should move this project very far forward.

Here is the PEP for multiple interpreters.
https://www.python.org/dev/peps/pep-0554/

It was rumored to be in Python3.7. Next I heard it would be in 3.8, then 3.9, now the Pep says provisional in 3.8, and they will make a decision about 3.9. I would not be surprised if multiple interpreters shipped first in gPython.

And I suspect that the cPython version will be one interpreter per OS thread. With gPython we could have thousands of interpreters.

I should also mention stackless.
https://github.com/stackless-dev/stackless/wiki

And finally the documents say that this is 1/5 th as fast as cPython. But I can easily get a server with 32 cores. And many applications now have little processing and lots of network I/O.

I just wonder what the killer app for this would be? It would be great to have a game, where a conference programming competition would be to win the multiplayer game, much like the snake competition at some recent Python conferences.

Warm Regards
Christopher Lozinski

given an ast, how does one generate the corresponding python source code

thank you

During I am writing a proposal for boxed runtime type.
I found that gpython does not support it yet.

len(range(255))

But accepted in gpython
it should be raised Syntax error.

cpython

>>> def a(x=3, b, c):
...     pass
...
  File "<stdin>", line 1
SyntaxError: non-default argument follows default argument

gpython

>>> def a(x=3, b, c):
...     pass
...
>>> a()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
TypeError: "a() missing 2 required positional arguments: 'x' and 'b'"

Current Results
in gpython

>>>> d_list = {'a' : 1, 'b' : 2}
>>>> for k, v in d_list.items("hi"):
>>>>     print(k,':',v)
a : 1
b : 2

in python

>>>> d_list = {'a' : 1, 'b' : 2}
>>>> for k, v in d_list.items("hi"):
>>>>     print(k,':',v)
Traceback (most recent call last):
  File "test.py", line 2, in <module>
    for k, v in d_list.items("hi"):
TypeError: items() takes no arguments (1 given)

I think we should add an error to this part of the code.

See #49

https://github.com/go-python/gopy

https://github.com/DataDog/go-python3

Might prove useful.

right now:

>>> import math
>>> help(math)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
NameError: NameError: "name 'help' is not defined"

In standard python

>>> x={"hello" : "world"}
>>> "hello" in x
True

In gpython

>>> x={"hello" : "world"}
>>> "hello" in x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
TypeError: "unsupported operand type(s) for iter: 'dict'"

Expected Result

>>> range(10)
range(0, 10)
>>> range(10) == range(10)
True
>>> str(range(10))
'range(0, 10)'

Actual Result

>>> range(10)
<range instance at 0xc0002c8380>
>>> range(10) == range(10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
TypeError: "unsupported operand type(s) for ==: 'range' and 'range'"
>>> str(range(10))
'<range instance at 0xc0002c8c60>'

The __repr__, __str__, __eq__, and __ne__ functions of the range object are not implemented.

Attribute errors occured using list.
​

Python (expected)

>>> a.pop()
4
>>>a
[1,2,3]
>>>a.pop(1)
2
>>>a
[1,3]

gpython (Actual)

>>> a = [1,2,3,4]
>>> a.pop()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
AttributeError: "'list' has no attribute 'pop'"

I read cpython twice. i think gpython is awesome, can i be a member of gpython? I will do my best.

Thanks to tengo and @ncw, we can compare the gpython with other interpreted which are implemented in Go.

https://github.com/d5/tengo#benchmark

fib(35) benchmark is slower than I expected, we can optimize this point later.
(tail call is faster than CPython, I can't believe it!)

>>> list(i for x in range(10))
[]

Which should have output

>>> list(i for x in range(10))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <genexpr>
NameError: name 'i' is not defined

An error should occur when returning a non-integer value from index.

Expected result (cpython)

>>> class C:
...     def __index__(self):
...         return 'a'
... 
>>> c = C()
>>> a = [1, 2 ,4]
>>> a[c]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __index__ returned non-int (type str)

Actual result (gpython)

>>> class C:
...     def __index__(self):
...         return 'a'
... 
>>> c = C()
>>> a = [1, 2, 4]
>>> a[c]
1

TODO

• Implement callable_iterator
• Implement iter() builtin

callable_itereator should be implemented before implementing iter() builtin

Currently the slice object is not fully implemented.
Many features, including instance creation, are not fully implemented and must be implemented.

Expected result (cpython)

>>> slice(10)
slice(None, 10, None)
>>> slice(10).indices(0)
(0, 0, 1)

Currently it's hard to close the REPL on Windows using standard terminal.
On WSL Ubuntu it works with Ctrl+Z, in Windows nothing happens.

This definition works:

class X(object):
  def __init__(self, x):
    self.x = x
  def __str__(self):
    return "my name is %s" % self.x

This doesn't:

class X(object):
  def __init__(self, x):
    self.x = x

  def __str__(self):
    return "my name is %s" % self.x

Both are accepted by C Python.

I want to try to implement hashlib (PR will follow).

For now I have implemented Type, module, module method and some "magic methods", but I cannot figure out how to implement object methods such as update, hexdigest etc.

Here is my WIP implementation.

Thanks in advance.

I'm currently implementing sorted, the signature is:

sorted(iterable, /, *, key=None, reverse=False)
    Return a new list containing all items from the iterable in ascending order.

    A custom key function can be supplied to customize the sort order, and the
    reverse flag can be set to request the result in descending order.

It already works as expected when passing either exactly one argument or all three.
When I only pass the first and the third, the third is interpreted as the second.

Python 3.4.0 (none, unknown)
[Gpython dev]
- os/arch: windows/amd64
- go version: go1.13
>>> l = [1.1, 1, 3]
>>> sorted(l)
[1, 1.1, 3]
>>> sorted(l, reverse=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
TypeError: "'bool' object is not callable"
>>> sorted(l, key=None, reverse=False)
[1, 1.1, 3]
>>> sorted(l, key=None, reverse=True)
[3, 1.1, 1]
>>>

I think my format string for py.ParseTupleAndKeywords should be "O|$Op:sorted", but this doesn't work too. Currently I have the following code:

func builtin_sorted(self py.Object, args py.Tuple, kwargs py.StringDict) (py.Object, error) {
	var iterable py.Object
	var keyFunc py.Object = py.None
	var reverse py.Object = py.False
	err := py.ParseTupleAndKeywords(args, kwargs, "O|Op:sorted", []string{"iterable", "key", "reverse"}, &iterable, &keyFunc, &reverse)
	if err != nil {
		return nil, err
	}
	l, err := py.SequenceList(iterable) // make a copy of the list
	if err != nil {
		return nil, err
	}
	err = SortInplace(l, keyFunc, reverse) // sort the copy in place
	if err != nil {
		return nil, err
	}
	return l, nil
}

Additionally Python doesn't allow sorted(list(), None, False), GPython allows this using the code above.

Got:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
TypeError: TypeError: 'all() takes exactly 1 argument (3 given)'

Expected:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
TypeError: 'all() takes exactly 1 argument (3 given)'

a = range(10)
d = a[10:11]
print(d)

Unlike the list type, the range type does not support the slice function. slice in the range type must be supported.

Expected result

range(10, 10)

Actual result

TypeError: "unsupported operand type(s) for index: 'slice'"
2019/09/11 23:10:28 TypeError: "unsupported operand type(s) for index: 'slice'"

Range object should support get_item operation :)

gpython

>>> a = range(10)
>>> a[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
TypeError: "'range' object is not subscriptable"

python3

Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> a = range(10)
>>> a[0]
0

Dict.ne doesn't return NotImplemented.

If result of __eq__ is Not Implemented, __ne__ should return NotImplemented, not True

Expected result (python)

>>> a = {}
>>> a.__ne__(3)
NotImplemented

Actual result (gpython)

>>> a = {}
>>> a.__ne__(3)
True

Printing a complex number displays the py.Complex type instead of the real+imag values:

>>> print(3+1j)
<complex64 instance at %!p(py.Complex=(3+1i))>

I added the M__exit__ method to file to support "with open() as f" but the execution fails as following:

>>> with open("testfile") as f:
...   print("hello")
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
AttributeError: AttributeError: "'file' has no attribute '__exit__'"

The code in this case is only checking for the exit attribute instead of also checking for the implemented interface.

I see this comment:

 // exit := py.ObjectLookupSpecial(mgr, "__exit__")

But I am not sure of how ObjectLookSpecial could be implemented.

An easier way to do this, but it feels "wrong", would be to add an exit method in the object dictionary.

There are some attributes that should be implemented for gpython.

Expected

>>> a = True
>>> a.imag
0
>>> a.real
1
>>> a.conjugate()
1

Actual

>>> a = True
>>> a.image
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
AttributeError: "'bool' has no attribute 'image'"
>>> a.real
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
AttributeError: "'bool' has no attribute 'real'"
>>> a.conjugate()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
AttributeError: "'bool' has no attribute 'conjugate'"

These operations should be implemented

>>> a = {1, 2, 3}
>>> b = {2, 3, 4, 5}
>>> a | b
{1, 2, 3, 4, 5}
>>> a & b
{2, 3}
>>> a - b
{1}
>>> a ^ b
{1, 4, 5}
>>>

a = 3
global a
print(a)

b = 2 + 5
print(b)

This code generate syntax error, but gpython doesn't stop program

Expected Result

Hyeockz:bin hyeockjinkim$ python3 g.py 
  File "g.py", line 2
    global a
SyntaxError: name 'a' is assigned to before global declaration

Actual Result

2019/09/09 00:08:36 name 'a' is assigned to before global declaration
3
7

SyntaxError in python3 is Error, not warning. So I think it should be modified.

TypeError occurs when comparing NotImplemented value from execution result.

Expected result (cpython)

>>> range(0).__ne__(2) == NotImplemented
True

Actual result

>>> range(0).__ne__(2) == NotImplemented
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
TypeError: "unsupported operand type(s) for ==: 'NotImplementedError' and 'NotImplementedError'"

The builtin print() method is currently printing values like this (see https://github.com/go-python/gpython/blob/master/builtin/builtin.go#L192):

fmt.Printf("%v", v)

The result is that some python objects are printed differently than expect:

None: print {}
True: print true
False: print false

The print method should probably use str or repr when available. As an alternative all types implementing str should also implement String()

Even if sep is not required, the option must be inserted.

gpython

with open("testfile","w") as f:
    print("1234", end = "@@@", file = f)
with open("testfile","r") as f:
    print(f.read())`

Traceback (most recent call last):
  File "test.py", line 3, in <module>
    FIXME line of source goes here
TypeError: 'print() argument 2 must be str, not file'
2019/09/26 11:21:21 TypeError: 'print() argument 2 must be str, not file'

when fixed code

with open("testfile","w") as f:
    print("1234", sep = '', end = "@@@", file = f)
with open("testfile","r") as f:
    print(f.read())

1234@@@

The following code writes different string to the stream than in CPython:

print(1,2,3, end="||\n")

python

>>> a = 10.0
>>> type(a)
<class 'float'>
>>> a.is_integer()
True

gpython

>>> a = 10.0
>>> type(a)
<class 'float'>
>>> a.is_integer()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
AttributeError: "'float' has no attribute 'is_integer'"

is_integer() method does not exist.

starting with Go-1.12, there's a nifty function runtime/debug.ReadBuildInfo to extract build informations out of a Go binary.

we could leverage this to display:

$> gpython
Python 3.4.0 (none, unknown)
[Gpython (devel)]
- os/arch: linux/amd64
- go version: devel +4f13a9c5b1 Tue Oct 1 07:16:47 2019 +0000
>>> 

(notice the (devel) string that comes out of debug.ReadBuildInfo().Main.Version

this could be factored into a gpython/vm.Version() function:

// Copyright 2019 The go-python Authors.  All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

// +build go1.12

package vm

import (
	"fmt"
	"runtime/debug"
)

// Version returns the version of Gpython and its checksum. The returned
// values are only valid in binaries built with module support.
//
// If a replace directive exists in the Gpython go.mod, the replace will
// be reported in the version in the following format:
//  "version=>[replace-path] [replace-version]"
// and the replace sum will be returned in place of the original sum.
//
// The exact version format returned by Version may change in future.
func Version() (version, sum string) {
	b, ok := debug.ReadBuildInfo()
	if !ok {
		return "", ""
	}

	const root = "github.com/go-python/gpython"

	modules := append([]*debug.Module{&b.Main}, b.Deps...)

	for _, m := range modules {
		if m.Path == root {
			if m.Replace != nil {
				switch {
				case m.Replace.Version != "" && m.Replace.Path != "":
					return fmt.Sprintf("%s=>%s %s", m.Version, m.Replace.Path, m.Replace.Version), m.Replace.Sum
				case m.Replace.Version != "":
					return fmt.Sprintf("%s=>%s", m.Version, m.Replace.Version), m.Replace.Sum
				case m.Replace.Path != "":
					return fmt.Sprintf("%s=>%s", m.Version, m.Replace.Path), m.Replace.Sum
				default:
					return m.Version + "*", m.Sum + "*"
				}
			}
			return m.Version, m.Sum
		}
	}

	return "", ""
}

Likewise, PyPy3, if we can display build information such as commit information, go version.
it will be awesome.

Python 3.5.3 (fdd60ed87e941677e8ea11acf9f1819466521bf2, Apr 26 2018, 01:23:42)
[PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Also, we need to implement copyright, creditsand license as a built-in attribute.

• copyright
• credits
• license
• build info

Expected

>>> a = {1, 2, 3}
>>> a
{1, 2, 3}

Actual

>>> a = {1, 2, 3}
>>> a
<set instance at 0xc0000aa018>

I installed gpython with the command go get github.com/go-python/gpython and when I run in the first time in repl I get the next message error:
Failed to open history: open /home/matheus/.gpyhistory: no such file or directory
the repl do not stop working, and not presented no problem at first, but should be nice dont show this kind of error since that this does not hassles the gpython experience repl

CPython:
repr([1,1.0]) # '[1, 1.0]'

GPython:
repr([1,1.0]) # '[1, 1]'

Hyeockz:bin hyeockjinkim$ ./gpython 
Python 3.4.0 (none, unknown)
[Gpython dev]
- os/arch: darwin/amd64
- go version: go1.11.4
>>> #
... 
... 
... 
... 
... 
... 

In python shell, #, ... is printed until input is received.
Since # is single-line comment, ... should be printed only once in shell.

The output in python3 is:

Python 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> #
... 
>>> 

as shown in #14, we should cross-check that our tests agree with some reference CPython version (presumably, CPython-3.)

so we should probably put some wiring so that each test.py script is run with both gpython and cpython.

Expected:

Gpython 3.4.0
>>> a = [1,2,3]
>>> a[0] = a
>>> a
[[...], 2, 3]

Actual behavior:

Gpython 3.4.0
>>> a = [1,2,3]
>>> a[0] = a
>>> a
runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow

in gpython

>>>> d = {'a' : 1}
>>>> d.get('a')
Traceback (most recent call last):
  File "test.py", line 13, in <module>
    FIXME line of source goes here
AttributeError: "'dict' has no attribute 'get'"
2019/09/29 16:24:04 AttributeError: "'dict' has no attribute 'get'"

in python

>>>> d = {'a' : 1}
>>>> d.get('a')
1

Expected

>>> def f():
...    f.x = 3
...    return f.x
...
>>> f()
3
>>>

Actual

>>> def f():
...    f.x = 3
...    return f.x
...
>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
  File "<stdin>", line 3, in f
    FIXME line of source goes here
SystemError: 'nil Dict in function'

As we know, CPython implemention has a GIL.
How about gpython?

right now, gpython fails with:

>>> import math
>>> dir(math)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    FIXME line of source goes here
NameError: NameError: "name 'dir' is not defined"

From python documentation at https://docs.python.org/3/library/functions.html#all
Return True if all elements of the iterable are true (or if the iterable is empty).