Computer Science, Mathematics. View 4 excerpts, cites methods. Boosting is one of the most significant advances in machine learning for classification and regression.
In its original and computationally flexible version, boosting seeks to minimize empirically a … Expand. View 9 excerpts, cites methods and background. Conjugate direction boosting for regression. Boosting in the context of linear regression has gained additional attraction by the invention of least angle regression LARS , where the connection between the lasso and forward stagewise fitting … Expand.
View 4 excerpts, cites background and methods. We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as … Expand. View 7 excerpts, cites methods and background. Special Invited Paper-Additive logistic regression: A statistical view of boosting. Boosting is one of the most important recent developments in classification methodology. Boosting works by sequentially applying a classification algorithm to reweighted versions of the training data … Expand.
View 3 excerpts, cites background. Boosting with early stopping: Convergence and consistency. View 3 excerpts, cites results and background. Gradient and Newton Boosting for Classification and Regression. Expert Syst. View 2 excerpts, cites background and methods. Additive Logistic Regression : a Statistical. View 1 excerpt, references methods. View 3 excerpts, references background. The list contains pointers to the During the retrieve step, the index is traversed from the ellipsoids.
During the retrieve step, it is scanned, starting root by following the subtree corresponding to the side of with the most recently used ellipsoid. This continues until the cutting plane that the query point lies on. Like the either an ellipsoid is found that contains the query point, Point Rtree, the Binary Tree requires no update when an or Ellr ellipsoids have been examined unsuccessfully.
If a ellipsoid is grown, because it only indexes the ellipsoid cen- containing ellipsoid is found, it is moved to the front of the ter points. A more detailed discussion of the index can be list. We therefore include tion. Instead of scanning the list for growable ellipsoids, a list-based data structure.
This simple structure has the we perform a NN-search for the query point to find grow advantage that if there is locality of access, we can directly candidates sorted by the Euclidean distance to the ellipsoid apply existing cache-replacement policies. The MRU List centers.
The search terminates once the Ellg nearest neigh- stores the ellipsoids ordered by their most recent access. The bors have been examined. All growable ellipsoids are grown. To improve the search for grow- update the Rtree. This tree is identical to the Point Rtree de- An add operation adds the new ellipsoid to the front of scribed above, but it is not used for the retrieve operation. Its center point, together with a pointer to the Notice that the leaf objects also contain a pointer to the ellipsoid object, is inserted into the Rtree.
We first examine the effect of tuning Ellr, the number of 5. Limiting the grow search to tures. Due to space constraints, we only discuss the two nearest neighbors was found to be enough to find all grow- best-performing indexes in detail and report the overall per- able ellipsoids.
We report the total number of retrieves Fig- formance for the others. As performance after tuning. If we do not tune any index, i. At the same time the number of retrieves in- outperforms the Binary Tree. However, the Binary Tree creases Figure 6 , because we are searching further, thereby benefits much more from tuning. This will be discussed in reducing Cmiss Effect 2. As we increase Ellr, Effect 2 ini- more detail in this section. However, at For our experiments we do not examine the effect of some point further increasing Ellr will only add very few the Ngrowmax parameter.
At this point Effect 1 causes the overall sim- growable candidates. However, we currently do not have any ulation time to increase slightly. Notice also that Ellg of nearest neighbors examined for growing. In this experi- directly limits Nfound , and therefore we can control Ngrown ment there were no restrictions placed on Ellr. The Rtree through Ellg. Hence in our experiments we only study the examines ellipsoids for growing in nearest neighbor order.
Therefore, as we start to increase Ellg, larger ellipsoids are grown and the domain is indexed more aggressively. Figure 7: Time vs. Figure Time vs. It might be unbalanced, i. For now we focus on the tree in the top half of the fig- Scenario 1 Scenario 2 ure, together with the corresponding view of the data space showing the cutting planes and ellipsoids.
X We illustrate the retrieve step with query point q2. When we reach a leaf node, we test if the ellip- F Z soid in the leaf contains the query point. This process, i. Notice that ellipsoids can straddle cutting planes, e. If ellipsoids are straddling planes, then the Primary Retrieve can result of misses decreases Figure 9. The total simulation time in a false negative. For example, q3 lies to the right of X Figure 10 therefore decreases Effect 7. Increasing Ellg and so the Primary Retrieve fails even though there exists increases the chances of finding at least one growable ellip- an ellipsoid A containing it.
To overcome this problem the soid, hence reduces the number of adds Effect 9. Reduc- Binary Tree performs a Secondary Retrieve if the Primary tion in the number of adds causes the list size to decrease fails. The main idea of the Secondary Retrieve is to explore for larger values of Ellg. In the case of q3 , the failed Primary Ellg, the number of retrieves asymptotes Figure 8 because Retrieve ended in leaf B.
Nearby subtrees are explored by no additional ellipsoids are found for growing. There is no moving up a level in the tree and exploring the other side increase in retrieve time. However, the total simulation time of the cutting plane. Specifically, we first examine C after then increases because tgrowsearch increases with Ellg caused moving up to Y , C is in the unexplored subtree. Then the by Effect 4 Figure Index Type 0. Assume that in the example in Bbox Rtree Figure 12, ellipsoid B can be grown to include q4 , but C Random Projection Rtree Binary Tree default and A cannot.
Growing of B made it straddle hyper-plane Y. Figure 14 shows that as we increase Ellg, Primary Re- The alternative to growing B is illustrated on the bottom trieves are being replaced by Secondary Retrieves, while the right part of Figure 12 Scenario 2.
This is because increasing Ellg is Now we add a new ellipsoid F with center q4 to the index. The The explanation of the super-linear increase in retrieve time add step requires the expensive computation of F , but it is similar to that described for tuning Ellr.
As we increase will enable future query points near q4 to be found by a Ellg the miss cost also increases slightly see Figure 15 be- Primary Retrieve.
Overall the total simulation time As we can see from this example, tuning parameter Ellg first decreases because of the dominance of Effect 7, but affects the Binary Tree in its choice of scenario 2 over 1. Furthermore, this choice, i. The experiments show It is evident from the detailed analysis in Section 5.
We will also see that the effect of the tuning performance of the indexes and the overall function approx- parameters is very different for the Binary Tree as compared imation algorithm. Table 3 lists the over- number of ellipsoids examined during the Secondary Re- all running time of the Methane combustion simulation; the trieve phase. It times are for the indexes after tuning, unless explicitly stated can be seen from Figure 13 that as we increase Ellr, tsearch otherwise. This increase in the retrieve time is ac- the index size increases with lower error tolerance and hence companied by a reduction in miss time, which is caused by smaller ellipsoids.
In fact, it outperforms all competitors. We which starts dominating the total simulation time. Figure 11 established that the cause for this difference was the ability reveals the explanation: As we increase Ellr, Secondary Re- of the Binary Tree to achieve a large number of Primary Re- trieves and hence also Nfpos are increasing, because we are trieves because it partitions the space, rather than searching searching the index more extensively.
Therefore we are re- through levels of overlapping bounding boxes. Careful tun- ducing the number of add operations, ultimately causing the ing can bias the Binary Tree toward a high rate of Primary Primary Retrieve rate to decrease Effect 3.
At the same Retrieves, with little reduction in overall retrieval rate. On time, the average cost of a Secondary Retrieve also increases, the other hand, tuning had comparatively little effect on the because the search proceeds further in the tree. These two Bbox Rtree. The overlap of bounding boxes at all levels of effects together—increase in number of Secondary Retrieves the tree resulted in large numbers of false positives during and in average cost per Secondary Retrieve—create the su- search.
We note here that the difference in performance of perlinear trend of the retrieve time with increasing Ellr. Effect 6 does not significantly affect performance because As we start increasing Ellg, because of Effect 7 the total very few ellipsoids actually grow during a single grow step.
Therefore The dramatic difference between the FIFO List and MRU there are fewer misses, which results in lower miss cost and List indexes is caused by locality in the combustion simula- better total simulation time see Figure Note the initial tion. Both index structures are identical; they only differ in drop in retrieve time in Figure The reason is that tsearch the order of the ellipsoids in the list.
MRU sorts by most re- includes the cost of all searches, including unsuccessful ones. Time Retrieves Time Total Time 5.
Ellr Binary Figure Number of retrieves vs. Since it does not know the spatial extent of the ellip- function approximation. For the combustion simulation, soids, we implemented the retrieve operation with an NN- neural networks have been proposed and used [14]. Unfortunately ever, there has been very little work on studying function because of the limited pruning power and the high cost of approximation as an indexing problem.
Our work is the first principled analysis uses the same Rtree during the grow step to find grow can- of the indexing problem.
Hence the performance difference between it and A large variety of index structures have been proposed the Point Rtree approach is mostly due to the poor retrieve by the database and computational geometry communities, performance of the Point Rtree.
Work prior to is surveyed in [5] Rtree to explore ways to improve its performance. Both are and [10]. In the following we discuss a few selected indexes, motivated by the problem that in high dimensions, hyper- which are most related to the ones studied in this paper. The bounding boxes contain a large fraction in the database community. It is thus a natural choice for The Random Projection Rtree addresses the problem indexing Local Regions for function approximation.
Several by projecting all ellipsoids onto a fixed set of k randomly se- variants of the Rtree have been proposed, e. The goal of most improvements a k-dimensional hyper-rectangle in the transformed space is to reduce the overlap of bounding boxes in tree nodes, defined by the projection lines. We can now use a stan- which is a major factor in degrading performance for high- dard Rtree to index the objects.
By using larger numbers dimensional data. The SS-Tree [28] takes this a step further of projections, we can achieve a tighter bounding polyhe- by using spheres as bounding regions. It is therefore a good dron around an ellipsoid, at the cost of more expensive in- candidate for managing spherical or ellipsoidal regions of dex operations in the higher-dimensional space. The results accuracy.
A BSP tree [10]. The Binary Tree used in our experiments detailed study of Random Projection Rtrees is part of our is an adaptation of this index structure. It has been shown that in high dimensions linear scans We can also reduce dead space by using ellipsoids as the are sometimes faster than complex index structures, espe- bounding shape at all tree levels. The corresponding El- cially when data is accessed on disk.
The VA-file [27] im- lipsoid Rtree performed very poorly, because of the high proves the performance of linear scans by quantizing the cost of basic index operations like testing if a point is within space. A simple approach based on scanning files at differ- a bounding ellipsoid or splitting nodes and computing the ent resolutions has been shown to outperform sophisticated new bounding ellipsoids, which is done approximately.
The Random Projection Rtree in this paper was mo- 6. The ISAT function approximation approach was first in- Multidimensional problems can be mapped to lower dimen- troduced by Pope [22]. It is one of the most widely used tech- sions by hashing [11, 15]. Machine duction and principal component analysis, e. Gionis, P. Indyk, and R. Similarity In this paper we introduced the function approximation search in high dimensions via hashing.
We showed its hardness and how it motivates Journal, pages —, A principled analysis of [12] A. R-trees: A dynamic index structure for the indexing problem led to the discovery of novel tradeoffs spatial searching.
Hastie, R. Tibshirani, and J. The structures. Elements of Statistical Learning. Springer, This is the first in-depth study of the general indexing [14] J. Hedengren and T. In situ adaptive problem in high-dimensional function approximation and tabulation for real-time control. Industrial and there are several avenues for future research. Jagadish, B. Ooi, K. Thus, a policy P r needs to be 4. Experiments plug-in electric vehicles EV optimally into power In the context of this publication, empirical studies have grids, where the control of charging power is seen as been performed for different models provided by the advantageous for reaching optimal power grid operation IEEE test case archive.
For illustration reasons, two of Clement, ; Clement, ; Sortomme, Out of these benchmark the individual behavior that mainly characterizes instances, dynamic problems are built according to electric vehicle charging load. Hutterer, c for learning policies for dynamic This PEV-charging control problem represents a optimization. In order to validate the approximated dynamic optimization task that requires optimal control policies, a test-procedure has been created that is based actions for high amounts of distributed EV-agents.
Therefore, within the Therefore, building a policy-based approach is the simulation of the dynamic power grid models, discrete- fundament of this show case. Here, each agent EV time states are expressed that represent one single state receives a flexible policy rather than static control of the system each. For these states, the deterministic decisions that makes it react to its environment OPF solution is computed with interior point method in dynamically, but in a globally optimal manner.
This MATPOWER and compared to actions that obtained policy is principally the same for all agents, but using best found policies lead to within these states. Here, a simulation model is built that represents a fleet of EVs within a given power 3. Results grid area, which will be integrated into the simulation- Table 1 lists the quality in means of error between the based evolutionary optimization of PEV charging OPF fitness function value of the best found policy control policies.
Policy Formulation For both benchmark instances, it can be shown that the A general policy shall be obtained. Its aim is to output a approximate policy-based control leads to near-optimal charging decision to an electric vehicle given the state decisions, that are in mean only 0.
This optimized worse than the reference solutions. Here, three 2 0. Mean Relative 0. However, over all EVs the system- the distribution grid, totally expected supply wide constraints are satisfied that enable secure power from renewables or financial aspects grid operation, while the objective function considering considering costs of electrical power supply.
Out of these classes of input data, in the context of this work, the authors defined once more a set of abstract rules r that gathers all needed information for decision making and provides compressed information to control units EVs in this case. Out of these rules, the general policy P r shall be learned that gives the decision on the charging power to a certain EV in the system.
Within this feeder, EVs are simulated to act individually. Additionally, renewable sources wind-power and photovoltaics are added to the system in order to create a dynamic and volatile scenario. A finally obtained charging policy has to derive robust charging decisions that provide system- wide near-optimal charging control over time.
Figure 2: Charging Power in Simulated Test Scenario Detailed discussions on this problem scenario as well as However, this example shall demonstrate the evolution formal definitions are provided in Hutterer, b. More detailed approximation to suchlike problems shall be illustrated. Results while this paper aims at providing a general view on the In this scenario, a general policy PEV r needs to be developed techniques. An outlook shall now depict computed that derives accurate control actions for each several open issues.
Equation 1 illustrates such a 5. However, this advantage only holds in a restrictive manner: A policy While the used input variables are discussed extensively is able to make accurate decisions within situations that in Hutterer, ; Hutterer, b , this policy are sufficiently similar to those situations it has been considers an EVs remain time at a charging spot ERT , trained to.
For other situations, its extrapolation-ability the actual electricity price AP , the actual wind speed is necessary to still make good decisions. As soon as AWS , the mean charging rate of all other EVs during specific situations are too different from the training the previous time step MCR , the past base load PBL simulation, obviously the policy-based control becomes at the previous time step as well as the actual solar useless.
In such a case, the simulation model would irradiance AI. Hence, if being learned accurately, policy-based control Figure 2 depicts the mean charging power over all is valid for systems where their behavior, dynamics and EVs in the simulated system when applying the evolved uncertainty are adequately predictable.
0コメント