194 Commits

Author SHA1 Message Date
Dylan K. Taylor
212d72657a
AsyncPool: Switch to a more efficient collection algorithm, revamp internals (#2493)
This greatly improves GC performance by being more intelligent about how it collects garbage tasks. It knows that if X task in the queue is not finished, none of the tasks behind it can be finished either, so there's no point checking them.

This also presents the opportunity to cleanup a lot of async pool internals, so I've taken it and torched a lot of garbage.
2018-10-25 15:36:38 +01:00
Dylan K. Taylor
90482e79bc Merge branch 'release/3.4' 2018-10-21 18:23:54 +01:00
Dylan K. Taylor
45c9caa38c Fixup some formatting issues 2018-10-21 18:15:25 +01:00
Dylan K. Taylor
d583dc25c3 Merge branch 'release/3.3' 2018-10-10 13:45:31 +01:00
Dylan K. Taylor
2858db430e Fixed AsyncTask publishProgress() race condition on task exit
It's possible for a progress update to be lost due to the task finishing before the main thread found the progress update.
2018-10-10 13:41:15 +01:00
Dylan K. Taylor
24677e1d79 Prevent undefined behaviour when accessing async worker thread-store from outside the worker itself 2018-10-08 13:01:12 +01:00
Dylan K. Taylor
a0f3c03b50 oopsie woopsie
I made a fucky wucky! a wittle fucko boingo!
2018-10-08 12:59:47 +01:00
Dylan K. Taylor
6ac0c517f5 Improve and simplify AsyncTask thread-local storage
This is now self-maintaining and doesn't rely on the async pool to wipe its ass on task completion. Instead, the garbage collector will cause thread-local data to be automatically released when the async task is garbage-collected on the main thread.
2018-10-08 12:48:44 +01:00
Dylan K. Taylor
e0574008b4 AsyncPool: Move removeTasks() functionality inside shutdown()
there's no need for a loop if we just have the workers shutdown directly, because they'll block until all tasks are done anyway.
2018-09-12 18:08:42 +01:00
Dylan K. Taylor
37190c9a65 Apply typehints to all AsyncTask methods
Since we're breaking API here anyway, no point in holding back on this.
2018-09-10 15:54:01 +01:00
Dylan K. Taylor
d62e00cc74 AsyncPool: Remove Server dependency (API breaks included)
This brings two plugin-breaking changes: AsyncTask->onCompletion() and AsyncTask->onProgressUpdate() no longer accept Server parameters.

This now allows for the functionality of AsyncPool and AsyncTask to be tested outside of a Server.
2018-09-10 15:48:59 +01:00
Dylan K. Taylor
83077309ef Merge branch 'release/3.2' 2018-08-04 16:03:46 +01:00
Dylan K. Taylor
f3f229ef7c Internet: only catch InternetExceptions - anything else is an unexpected fault condition 2018-08-04 14:51:26 +01:00
Dylan K. Taylor
779d92c656 BulkCurlTask: Remove complexData parameter (BC break)
This is a leftover from when it was necessary to pass complex data to the AsyncTask constructor in order to have it locally-stored. Since this has now been superseded by storeLocal(), it doesn't make sense for this parameter to exist anymore.
2018-08-04 14:44:20 +01:00
Dylan K. Taylor
6614183c7f Merge branch 'release/3.1' into release/3.2 2018-08-03 20:07:52 +01:00
Dylan K. Taylor
87f458f9bd AsyncPool: remove now-unnecessary isTerminated() call 2018-08-03 20:07:37 +01:00
Dylan K. Taylor
5a7e575c3a AsyncPool: isCrashed() now returns true when a fatal error occurred
the fix for chunks earlier didn't fix...
2018-08-03 20:06:41 +01:00
Dylan K. Taylor
706c620d04
Move Internet-related functions from Utils into their own class (#2324)
- Added `Internet::getIP()`, `Internet::getURL()`, `Internet::postURL()`, and `Internet::simpleCurl()`.
- Deprecated the corresponding functions in `Utils`. Updating to the new functions is as simple as replacing `Utils` with `Internet`, since this doesn't break backwards compatibility.

The deprecations should be catered for by plugin developers. These deprecated redirects will be removed no later than 4.0.0.
2018-07-25 15:51:18 +01:00
Dylan K. Taylor
2e18fe710c MemoryManager: Shut down idle workers during GC to reclaim memory
workers can be a major memory hog, especially if you have lots of them.
2018-07-14 18:07:37 +01:00
Dylan K. Taylor
53068caf3c Level: Only register generators when attempting to actually generate chunks
This saves a ton of memory on servers which don't generate any chunks during their runtime (which is most servers).
2018-07-14 11:34:55 +01:00
Dylan K. Taylor
95787c2be9 TaskScheduler: fixed tasks not getting destroyed when all tasks in scheduler are cancelled before next heartbeat
they got removed from the tasks array, but not from the queue (for performance reasons). The queue gets cleaned up by the heartbeat, but it was checking if there were things in the main array, not in the queue.

There are a couple of other bugs with cancelling tasks that this doesn't fix that are rather more complicated to deal with.
2018-06-21 12:49:08 +01:00
Dylan K. Taylor
80daaf09b2 SendUsageTask: fixed oops x2 (thanks PhpStorm)
looks like this file was somehow missed during the refactor.
2018-06-18 12:08:55 +01:00
Dylan K. Taylor
da6439e3f4 SendUsageTask: fixed oops 2018-06-17 16:13:05 +01:00
Dylan K. Taylor
05af87e1d4 Strip empty lines at the end of classes 2018-06-11 13:19:23 +01:00
Dylan K. Taylor
3be6665e3d AsyncWorker IDs now start from 0 instead of 1
this now matches their IDs in the AsyncWorker pool, as used by submitTask().
2018-06-11 11:58:24 +01:00
Dylan K. Taylor
17607b8116 AsyncPool: added some void typehints 2018-06-11 10:41:40 +01:00
Dylan K. Taylor
4c98d9d3ad AsyncPool: add a type doc to $size 2018-06-11 10:40:01 +01:00
Dylan K. Taylor
7d5b3079bc AsyncPool: more documentation 2018-06-11 10:38:51 +01:00
Dylan K. Taylor
88d83e0fca AsyncPool: Lazy-start AsyncWorkers when they are needed only
This changes how the AsyncPool works so that it does not immediately always start all of the workers in the pool.
Instead, workers will be started only when an idle worker was not found.
This allows for significant memory footprint reductions while idle.

In effect the async-workers setting in pocketmine.yml now dictates a _maximum_ pool size, not a fixed pool size.
2018-06-11 10:23:46 +01:00
Dylan K. Taylor
3725bea3e5 AsyncPool: Slightly reduce worker memory usage with more conservative start options
this results in a memory footprint reduction of maybe 4MB for a total of 8 workers. Not much, but it's something.
2018-06-10 10:18:07 +01:00
Dylan K. Taylor
c327b3d2c4 AsyncPool: be less dependent on Server in the code
The goal is to remove the Server things from here completely.
2018-06-08 20:08:23 +01:00
Dylan K. Taylor
b331f8e1c9 AsyncWorker: added removeFromThreadStore()
and use it instead of overwriting with null things, which still occupies memory
2018-06-07 10:12:50 +01:00
Dylan K. Taylor
ee787974f2 AsyncWorker: Use statics for thread-local worker storage instead of globals 2018-06-07 09:59:32 +01:00
Dylan K. Taylor
1a21041d00 AsyncPool: Parameterize worker memory limit instead of calling back into Server 2018-06-06 18:00:56 +01:00
Dylan K. Taylor
35eaf38ca1 MemoryManager: parameterize an abstract \Logger to remove dependency on MainLogger 2018-06-04 13:30:48 +01:00
Dylan K. Taylor
1d71f0cf43 AsyncWorker: added getLogger() 2018-06-04 13:29:38 +01:00
Dylan K. Taylor
02b4eeeb9b TaskScheduler: Remove repeating tasks which throw exceptions 2018-05-31 14:10:59 +01:00
Dylan K. Taylor
6b4b4e4bb1 TaskScheduler: remove redundant else branch 2018-05-31 13:01:06 +01:00
Dylan K. Taylor
f2b8d6879f TaskScheduler: remove leftover code that makes no sense
this is impossible now that we have typehints
2018-05-31 12:52:29 +01:00
Dylan K. Taylor
60212cef2f TaskScheduler: adjust disabled scheduler exception 2018-05-31 12:38:36 +01:00
Dylan K. Taylor
15270f8329 Fixed plugin schedulers crashing after disable/reenable 2018-05-31 12:34:22 +01:00
Dylan K. Taylor
05ef13b23a Merge branch 'master' into scheduler-nuke
# Conflicts:
#	src/pocketmine/level/Level.php
2018-05-31 10:40:10 +01:00
Dylan K. Taylor
1b4723d816 AsyncWorker: don't require an attachable logger 2018-05-31 10:34:42 +01:00
Dylan K. Taylor
b3043f9552 Task: remove obsolete doc comment
not anymore sunshine
2018-05-30 21:30:43 +01:00
Dylan K. Taylor
18fdbc2834 TaskScheduler: fix typo in isReady() parameter 2018-05-30 16:45:37 +01:00
Dylan K. Taylor
a8c766be88 Remove TaskHandler dependency on MainLogger
This instead allows the exception to be caught by the scheduler and reported using its logger.
2018-05-30 16:00:25 +01:00
Dylan K. Taylor
e20be3eeba Move task timings responsibility from scheduler to handler 2018-05-30 15:38:39 +01:00
Dylan K. Taylor
51f43fb375 Removed global ServerScheduler - plugins now get their own isolated schedulers
This change breaks pretty much all API pertaining to synchronous task scheduling.

Significant changes:
- Server->getScheduler() has been removed
- Plugin->getScheduler() has been added - every plugin now has its own scheduler
- Because schedulers are now per-plugin, it is now unnecessary for PluginTask to exist because stopping plugin tasks on plugin disable is as simple as destroying the plugin's scheduler. Therefore PluginTask has now been removed and it is expected for things to now use the base Task class instead.

For the most part, plugins will simply need to change Plugin->getServer()->getScheduler()->... to Plugin->getScheduler()->...
Another highlight is that plugin tasks now no longer have global IDs - they are unique to each scheduler.
2018-05-30 14:11:11 +01:00
Dylan K. Taylor
132746aa3d ServerScheduler: Require a Logger instance as ctor param, now non-dependent on Server
yay for unit-testing and reusability!!!
2018-05-30 12:29:19 +01:00
Dylan K. Taylor
d03f36ebee First look at splitting up AsyncPool and ServerScheduler
This commit contains quite a few breaking changes with respect to how AsyncTasks are handled. This is necessary to allow separation of the ServerScheduler and the AsyncPool, because in the future the ServerScheduler may be removed and instead there will be isolated per-plugin sync-task schedulers - but we cannot have every plugin with its own worker pool for memory usage reasons if nothing else.

The following things have changed:
- ServerScheduler: scheduleAsyncTask(), scheduleAsyncTaskToWorker(), getAsyncTaskPoolSize(), increaseAsyncTaskPoolSize() and similar methods have all been removed. Additionally the static \$WORKERS field has been removed.
- Server: added API method getAsyncPool(). This grants you direct access to the server's AsyncPool. Calls to getScheduler()->scheduleAsyncTask() and scheduleAsyncTaskToWorker() should be replaced with getAsyncPool()->submitTask() and submitTaskToWorker() respectively.
2018-05-30 12:20:10 +01:00