Prog Lang and Data Structure

Mach Learning and Data Mining

2014_Final.pdf

CSC 192 Lecture 10 Admin Stuff 1) Assignment 2 is now up. Due date is on 11 th October 2011. 2) I hope to post Assignment 1 grades by 10 th October. 3) There is no lab this week. Functions ● A function is a series of statements that have been grouped together and given a name. ● Each function is essentially a small program, with its own declarations and statements. ● Advantages of functions: ● A program can be divided into small pieces that are easier to understand and modify. ● We can avoid duplicating code that’s used more than once. ● A function that was originally part of one program can be reused in other programs. ● Before we go over the formal rules for defining a function, let’s look at three simple programs that define functions. ● A function named average that computes the average of two double values: double average(double a, double b) { return (a + b) / 2; } ● The word double at the beginning is the return type of average . ● The identifiers a and b (the function’s parameters ) represent the numbers that will be supplied when average is called. ● Every function has an executable part, called the body, which is enclosed in braces. ● The body of average consists of a single return statement. ● Executing this statement causes the function to “return” to the place from which it was called; the value of (a + b) / 2 will be the value returned by the function. ● A function call consists of a function name followed by a list of arguments. – average(x, y) is a call of the average function. ● Arguments are used to supply information to a function. ● The call average(x, y) causes the values of x and y to be copied into the parameters a and b . ● An argument doesn’t have to be a variable; any expression of a compatible type will do. – average(5.1, 8.9) and average(x/2, y/3) are legal. ● We’ll put the call of average in the place where we need to use the return value. ● A statement that prints the average of x and y : printf("Average: %g\n", average(x, y)); The return value of average isn’t saved; the program prints it and then discards it. ● If we had needed the return value later in the program, we could have captured it in a variable: avg = average(x, y); ● The average.c program reads three numbers and uses the average function to compute their averages, one pair at a time: Enter three numbers: 3.5 9.6 10.2 Average of 3.5 and 9.6: 6.55 Average of 9.6 and 10.2: 9.9 Average of 3.5 and 10.2: 6.85 average.c /* Computes pairwise averages of three numbers */ #include <stdio.h> double average(double a, double b) { return (a + b) / 2; } int main(void) { double x, y, z; printf("Enter three numbers: "); scanf("%lf%lf%lf", &x, &y, &z); printf("Average of %g and %g: %g\n", x, y, average(x, y)); printf("Average of %g and %g: %g\n", y, z, average(y, z)); printf("Average of %g and %g: %g\n", x, z, average(x, z)); return 0; }

CSC | 192 | Prog Lang and Data Structure | lecture notes | CSC 192 Lecture 10 Admin Stuff

07_multiclass.pdf-CSC 411: Lecture 07: Multiclass Classiﬁcation

Minimum Spanning Tree in black Prims Algorithm to find MST T = a spanning tree containing a single node s; E = set of edges adjacent to s; while T does not contain all the nodes { remove an edge (v, w) of lowest cost from E if w is already in T then discard edge (v, w) else { add edge (v, w) and node w to T add to E the edges adjacent to w } }        References: 3 rd Edition, Algorithms in C by Robert Sedgewick

CSC | 192 | Prog Lang and Data Structure | lecture notes | Minimum Spanning Tree in black

CSC 411: Lecture 18: Ensemble Methods II Raquel Urtasun & Rich Zemel University of Toronto Nov 23, 2015 Urtasun & Zemel (UofT) CSC 411: 18-Ensemble Methods II Nov 23, 2015 1 / 13 Today Random/Decision Forest Mixture of Experts Urtasun & Zemel (UofT) CSC 411: 18-Ensemble Methods II Nov 23, 2015 2 / 13 What are the base classiﬁers? Popular choices of base classiﬁer for boosting and other ensemble methods: I Linear classiﬁers I Decision trees Urtasun & Zemel (UofT) CSC 411: 18-Ensemble Methods II Nov 23, 2015 3 / 13 Random/Decision Forests Deﬁnition: Ensemble of decision trees Algorithm: I Divide training examples into multiple training sets (bagging) I Train a decision tree on each set (can randomly select subset of variables to consider) I Aggregate the predictions of each tree to make classiﬁcation decision (e.g., can choose mode vote) Urtasun & Zemel (UofT) CSC 411: 18-Ensemble Methods II Nov 23, 2015 4 / 13 Ensemble learning: Boosting and Bagging Experts cooperate to predict output … outp ut Σ Expert 1 Expert 2 Expert M g 1 g 2 y( x ) x Vote of each expert has consistent weight for each test example y ( x )= X m g m y m ( x ) Urtasun & Zemel (UofT) CSC 411: 18-Ensemble Methods II Nov 23, 2015 5 / 13 Mixture of Experts Weight of each expert is not constant – depends on input x … y( x ) x Σ g 1 (x) g 2 (x) Expert 1 Expert 2 Expert M Ga&ng Network Gating network encourages specialization (local experts) instead of cooperation y ( x )= X m g m ( x ) y m ( x ) Urtasun & Zemel (UofT) CSC 411: 18-Ensemble Methods II Nov 23, 2015 6 / 13 Mixture of Experts: Summary 1. Cost function designed to make each expert estimate desired output independently 2. Gating network softmax over experts : stochastic selection of who is the true expert for given input 3. Allow each expert to produce distribution over outputs Urtasun & Zemel (UofT) CSC 411: 18-Ensemble Methods II Nov 23, 2015 7 / 13 Cooperation vs. Specialization Consider regression problem To encourage cooperation, we can train to reduce discrepancy between average of predictors with target E =( t - 1 M X m y m ( x )) 2 This can overﬁt badly. It makes the model much more powerful than training each predictor separately Leads to odd objective: consider adding models/experts sequentially I if its estimate for t is too low, and the average of other models is too high, then model m encouraged to lower its prediction Urtasun & Zemel (UofT) CSC 411: 18-Ensemble Methods II Nov 23, 2015 8 / 13

All courses in CSC

Popular Documents