Pony ORM Release 0.7.6

In release 0.7.4 we introduced support of hybrid methods and properties. This change required massive refactoring which lead to some bugs. Current release fixes all new bugs revealed after 0.7.4.

Also in this release some nice features were added. The first is the support of Literal String Interpolation a.k.a f-strings, introduced in Python 3.6. It looks nicely than using of %-interpolation or format method. f-strings use two special bytecode instructions which Pony now able to decompile and translate correctly:

select(f'The age of {s.name} is {s.age}' for s in Student)

The second small feature is a possibility to skip initial rows in query by specifying offset without limit. In order to do this, you can pass None as a limit value, or just omit limit value at all.

Also now it is possible to explicitly case JSON expressions to int, str or float. It may be necessary if you want to mix JSON expressions with expressions of other types in functions like coalesce()

Finally, we introduced @db.on_connect decorator which can be used to set up some connection properties. For example, since release 0.7.4 like in SQLite works as case-sensitive for better compatibility with other database management systems. If for some reason you want to restore the old like behavior, you can set corresponding pragma on SQLite connection:

@db.on_connect(provider='sqlite')
def set_case_insensitive_like(db, connection):
    cursor = connection.cursor()
    cursor.execute('PRAGMA case_sensitive_like = OFF')

Below is the full changelog since 0.7.5:

Features

  • f-strings support in queries select(f'{s.name} - {s.age}' for s in Student)
  • #344: It is now possible to specify offset without limit: query.limit(offset=10)
  • #371: Support of explicit casting of JSON expressions to str, int or float
  • @db.on_connect decorator added

Bugfixes

  • Fix bulk delete bug introduced in 0.7.4
  • #370 Fix memory leak introduced in 0.7.4
  • Now exists() in query does not throw away condition in generator expression: exists(s.gpa > 3 for s in Student)
  • #373: 0.7.4/0.7.5 breaks queries using the in operator to test membership of another query result
  • #374: auto=True can be used with all PrimaryKey types, not only int
  • #369: Make QueryResult looks like a list object again: add concatenation with lists, .shuffle() and .to_list() methods
  • #355: Fix binary primary keys PrimaryKey(buffer) in Python2
  • Interactive mode support for PyCharm console
  • Fix wrong table aliases in complex queries
  • Fix query optimization code for complex queries
  • Fix a bug with hybrid properties that use external functions

Pony ORM Releases 0.7.4 & 0.7.5

In these releases we add support of Python 3.7 and PyPy. Also we changed PonyORM internals significantly in order to add two long-demanded features. The first is hybrid methods and properties. Now it is possible to define one-line method or property in entity class and use it inside declarative queries. This way it is possible to re-use complex expressions between different queries:

class Person(db.Entity):
    first_name = Required(str)
    last_name = Required(str)
    country = Required(str)
    age = Required(int)

    @property
    def full_name(self):
        return self.first_name + self.last_name

    @property
    def is_adult(self):
        return self.age > 18

...

query = select(p for p in Person if p.is_adult)

The second feature is the possibility to base query on a previous query. This way it is possible to construct complex queries gradually. In previous versions of Pony it was possible to add conditions by using query.filter method:

query2 = query.filter(lambda person: person.country == 'USA')

but query.filter function cannot change the result type of query. Now you have full flexibility for making your query:

query3 = select(x.full_name for x in query2 if x.last_name.startswith('A'))

Implementing this two features was not easy, because SQL and Python have different namespace rules, and mapping Python name to SQL aliases is a non-trivial task for complex queries.

Also we added a pony.flask subpackage wich provide integration with Flask, so extensions like Flask-Login can work with Pony.

From now on we will concentrate on migration tool and hopefully release it soon.

Here is the detailed changelog:

Pony ORM Release 0.7.4 (2018-07-23)

Major features

  • Hybrid methods and properties added
  • Allow to base queries on another queries: select(x.a for x in prev_query if x.b)
  • Added support of Python 3.7
  • Added support of PyPy
  • group_concat() aggregate function added
  • pony.flask subpackage added for integration with Flask

Other features

  • distinct option added to aggregate functions
  • Support of explicit casting to float and bool in queries

Improvements

  • Apply @cut_traceback decorator only when pony.MODE is ‘INTERACTIVE’

Bugfixes

  • In SQLite3 LIKE is case sensitive now
  • #249: Fix incorrect mixin used for Timedelta
  • #251: correct dealing with qualified table names
  • #301: Fix aggregation over JSON Column
  • #306: Support of frozenset constants added
  • #308: Fixed an error when assigning JSON attribute value to the same attribute: obj.json_attr = obj.json_attr
  • #313: Fix missed retry on exception raised during db_session.__exit__
  • #314: Fix AttributeError: ‘NoneType’ object has no attribute ‘seeds’
  • #315: Fix attribute lifting for JSON attributes
  • #321: Fix KeyError on obj.delete()
  • #325: duplicating percentage sign in raw SQL queries without parameters
  • #331: Overriding __len__in entity fails
  • #336: entity declaration serialization
  • #357: reconnect after PostgreSQL server closed the connection unexpectedly
  • Fix Python implementation of between() function and rename arguments: between(a, x, y) -> between(x, a, b)
  • Fix retry handling: in PostgreSQL and Oracle an error can be raised during commit
  • Fix optimistic update checks for composite foreign keys
  • Don’t raise OptimisticCheckError if db_session is not optimistic
  • Handling incorrect datetime values in MySQL
  • Improved ImportError exception messages when MySQLdb, pymysql, psycopg2 or psycopg2cffi driver was not found
  • desc() function fixed to allow reverse its effect by calling desc(desc(x))
  • __contains__ method should check if objects belong to the same db_session
  • Fix pony.MODE detection; mod_wsgi detection according to official doc
  • A lot of inner fixes

Pony ORM Release 0.7.5 (2018-07-24)

Bugfixes

  • query.where and query.filter method bug introduced in 0.7.4 was fixed

Pony ORM Release 0.7.3

In this release we have added the sort_by method of the Query object. At this point it is an alias of the order_by method. In the release 0.8 we are going to make a backward incompatible change and slightly modify the behavior of the order_by method. If you replace the order_by with sort_by it will continue working without any problems in this and future releases. There is no urgency to make this change – it is a preparation for the 0.8 release.

Here is the list of changes in this release:

New features

  • where() method added to the Query object
  • coalesce() function added
  • between(x, a, b) function added
  • #295: Add _table_options_ for entity class to specify engine, tablespace, etc.
  • Make debug flag thread-local
  • sql_debugging context manager added
  • sql_debug and show_values arguments were added to db_session
  • set_sql_debug function added as alias to (to be deprecated) sql_debug function
  • Allow db_session to accept ddl parameter when used as context manager
  • Add optimistic=True option to db_session
  • Skip optimistic checks for queries in db_session with serializable=True
  • fk_name option added for attributes in order to specify foreign key name
  • #280: Now it’s possible to specify timeout option in db.bind() method for SQLite, as well as pass other keyword arguments for sqlite3.connect function
  • Add support of explicit casting to int in queries using int() function
  • Added modulo division % native support in queries

Bugfixes

  • Fix bugs with composite table names
  • Fix invalid foreign key & index names for tables which names include schema name
  • For queries like select(x for x in MyObject if not x.description) add “OR x.info IS NULL” for nullable string columns
  • Add optimistic checking for delete() method
  • Show updated attributes when OptimisticCheckError is being raised
  • Fix incorrect aliases in nested queries
  • Correctly pass exception from user-defined functions in SQLite
  • More clear error messages for UnrepeatableReadError
  • Fix db_session(strict=True) which was broken in 2d3afb24
  • Fixes #170: Problem with a primary key column used as a part of another key
  • Fixes #223: incorrect result of getattr(entity, attrname) when the same lambda applies to different entities
  • Fixes #266: Add handler to "pony.orm" logger does not work
  • Fixes #278: Cascade delete error: FOREIGN KEY constraint failed, with complex entity relationships
  • Fixes #283: Lost Json update immediately after object creation
  • Fixes #284: query.order_by() orders Json numbers like strings
  • Fixes #288: Expression text parsing issue in Python 3
  • Fixes #293: translation of if-expressions in expression
  • Fixes #294: Real stack traces swallowed within IPython shell
  • Collection.count() method now checks if session is alive
  • Set obj._session_cache_ to None after exiting from db session for better garbage collection
  • Unload collections which are not fully loaded after exiting from db session for better garbage collection
  • Raise on unknown options for attributes that are part of relationship

Pony ORM Release 0.7.2

Here is the list of changes:

  • All arguments of db.bind() can be specified as keyword arguments. Previously Pony required the first positional argument which specified the database provider. Now you can pass all the database parameters using the dict: db.bind(**db_params). See Pony reference for more information
  • The optimistic attribute option is added – reference
  • Fixed #219: when a database driver raises an error, sometimes this error was masked by the ‘RollbackException: InterfaceError: connection already closed’ exception. This happened because on error, Pony tried to rollback transaction, but the connection to the database was already closed and it masked the initial error. Now Pony displays the original error which helps to understand the cause of the problem.
  • Fixed #276: Memory leak
  • Fixes the __all__ declaration. Previously IDEs, such as PyCharm, could not understand what is going to be imported by from pony.orm import *. Now it works fine.
  • Fixed #232: negate check for numeric expressions now checks if value is zero or NULL
  • Fixed #238, #133: raise TransactionIntegrityError exception instead of AssertionError if obj.collection.create(**kwargs) creates a duplicate object
  • Fixed #221: issue with unicode json path keys
  • Fixed bug when discriminator column is used as a part of a primary key
  • Handle situation when SQLite blob column contains non-binary value

To install the updated version use the following command:

pip install --upgrade pony

Pony ORM Release 0.7.1

This release brings Python 3.6 support and has some bugfixes.

Here is the list of changes:

  • Fixed #216: decompiler fixes for Python 3.6
  • Fixed #203: subtranslator should use argnames from parent translator
  • Change a way aliases in SQL query are generated in order to fix a problem when a subquery alias masks a base query alias
  • Volatile attribute bug fixed
  • Fix creation of self-referenced foreign keys – before this Pony didn’t create the foreign key for self-referenced attributes
  • Bug fixed: when required attribute is empty the loading from the database shouldn’t raise the validation error. Now Pony raises the warning DatabaseContainsIncorrectEmptyValue
  • New warning DatabaseContainsIncorrectEmptyValue added, it is raised when the required attribute is empty during loading an entity from the database
  • Throw an error with more clear explanation when a list comprehension is used inside a query instead of a generator expression: “Use generator expression (… for … in …) instead of list comprehension [… for … in …] inside query”

To install the updated version use the following command:

pip install --upgrade pony

Pony ORM Release 0.7

We are excited to announce that

Starting with this release Pony ORM is release under the Apache License, Version 2.0!

Previously we were using the AGPL and commercial licenses and found that it was driving away contributors and “open source zealots”. Pony is a mature library which is used by many companies in a production mode. We’d like to grow a community around Pony ORM and that is why we decided to release it under a permissive license, such as Apache 2.0. The Apache License allows you to freely use, modify, and distribute any Apache licensed product. You can use Pony ORM in your commercial products for free.

In this article you can find more information about the Apache license.

Besides this, here is the list of changes, improvements and bug fixes included into this release:

  • Added getattr() support in queries
  • Fixed 159: exceptions happened during flush() should not be wrapped with the CommitException
    Before this release an exception that happened in a hook , could be raised in two ways – either wrapped into the CommitException or without wrapping. It depended if the exception happened during the execution of flush() or commit() function on the db_session exit. Now the exception happened inside the hook never will be wrapped into the CommitException.
  • Fixed 190: Timedelta is not supported when using pymysql

To install the updated version use the following command:

pip install --upgrade pony

Pony ORM Release 0.6.6

The main feature of this release is native JSON data type support in databases. We have added a new chapter to the documentation, which describes how it works.

Here is the list of changes, improvements and bug fixes:

  • Starting with this release Pony drops Python 2.6 support
  • Added an experimental @db_session strict parameter
  • Merged #179: Added the compatibility with PYPY using psycopg2cffi
  • Fixed #182 – LEFT JOIN doesn’t work as expected for inherited entities when foreign key is None
  • Fixed some small bugs

To install the updated version use the following command:

pip install --upgrade pony

Pony ORM Release 0.6.5

This release contains a number of bug fixes and one enhancement.

  • Fixed #168: Incorrect caching when slicing the same query multiple times
    The bug appeared when the same query instance was sliced more than once. The result of first slicing was cached and used for all subsequent slices. The correct behavior is to generate new SQL query with different LIMIT…OFFSET parameters.
  • Fixed #169: When py_check() returns False, Pony should truncate too large values in resulting ValueError message
    Before this fix, when you tried to assign a long str or bytes value and if didn’t pass the validation, the whole string was printed in the stacktrace. That could be inconvenient if the string was pretty long. Now we show only first 100 symbols and truncate the rest.
  • Fixed #171: AssertionError when saving changes of multiple objects
    The bug appeared when there was a one-to-one relationship between two entities and both ends of this relationship were declared as Optional. When you tried to create objects and link them in the same transaction, it could cause the AssertionError: _save_() called for object ... with incorrect status 'inserted'
  • Fixed #172: Query prefetch() method should load specified lazy attributes right in the main query if possible
    Before this release, the result of prefetch() method on a lazy attribute was a separate SQL query. That’s because the goal was not in minimizing SQL queries, but in loading all necessary data before leaving the db_session scope. Now Pony does prefetch smarter – it tries to minimize the number of SQL queries.

    Let’s consider an example:

        from pony.orm import *
    
        db = Database()
    
        class Post(db.Entity):
            title = Required(str, 128)
            text = Required(LongStr)
    
        db.bind('sqlite', ':memory:')
        db.generate_mapping(create_tables=True)
        sql_debug(True)
    
        with db_session:
            posts = Post.select().prefetch(Post.text)[:]
    

    In this example the text attribute is declared as the lazy string type. Before this release Pony always loaded lazy attributes using a separate SQL query. Now, if you pass such a lazy attribute to the prefetch() method, Pony will retrieve this field in the main SQL query:

        SELECT "p"."id", "p"."title", "p"."text"
        FROM "Post" "p"
    
  • Fixed #176: Autostripped strings are not validated correctly for Required attributes
    The bug was that Pony automatically striped attribute value after validation, not before it. So, if we had a Required attribute and tried to assign some value which contained only spaces, Pony would through the ValueError: Attribute ... is required. Now Pony first strips the value and then does the validation.

To install the updated version use the following command:

pip install --upgrade pony

Pony ORM Release 0.6.4

This release brings no new features, has no backward incompatible changes, only bug fixes.
If you are using obj.flush() method in your code we recommend you to upgrade to 0.6.4 release.

  • Fixed #161: 0.6.3 + obj.flush(): after_insert, after_update & after_delete hooks do not work

To install the updated version use the following command:

pip install --upgrade pony

Pony ORM Release 0.6.3

This release doesn’t bring any new features, only bug fixes. The release brings no backward-incompatible changes, so you can upgrade from PonyORM 0.6.2 without any modifications in your code.

  • Fixed #138: Incorrect behavior of obj.flush(): assertion failed after exception
  • Fixed #157: Incorrect transaction state after obj.flush() caused “release unlocked lock” error in SQLite
  • Fixed #151: SQLite + upper() or lower() does not work as expected

#138: Incorrect behavior of obj.flush(): assertion failed after exception,
#157: Incorrect transaction state after obj.flush() caused “release unlocked lock” error in SQLite

These are long-standing bugs, but they were discovered just recently. The bugs were caused by incorrect implementation of obj.flush() method. In the same time the global flush() function worked correctly. Before this fix the method obj.flush() didn’t call before_xxx and after_xxx hooks. In some cases it led to assertion error or some other errors when using SQLite.

Now both bugs are fixed and the method obj.flush() works properly.

#151: SQLite + upper() or lower() does not work as expected

Since SQLite does not support Unicode operations, the upper() and lower() SQL functions do not work for non-ascii symbols.

Starting with this release Pony registers two additional unicode-aware functions in SQLite: py_upper() and py_lower(), and uses these functions instead of the standard upper and lower functions:

>>> select(p.id for p in Person if p.name.upper() == 'John')[:]

SQLite query:

SELECT "p"."id"
FROM "Person" "p"
WHERE py_upper("p"."name") = 'John'

For other databases Pony still uses standard upper() and lower() functions.