annotate lifelines_tool/test-data/readme_sample @ 0:dd49a7040643 draft

Initial commit
author fubar
date Wed, 09 Aug 2023 11:12:16 +0000
parents
children 232b874046a7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
dd49a7040643 Initial commit
fubar
parents:
diff changeset
1 ## Lifelines tool starting.
dd49a7040643 Initial commit
fubar
parents:
diff changeset
2 Using data header = Index(['Unnamed: 0', 'week', 'arrest', 'fin', 'age', 'race', 'wexp', 'mar',
dd49a7040643 Initial commit
fubar
parents:
diff changeset
3 'paro', 'prio'],
dd49a7040643 Initial commit
fubar
parents:
diff changeset
4 dtype='object') time column = week status column = arrest
dd49a7040643 Initial commit
fubar
parents:
diff changeset
5 Logrank test for race - 0 vs 1
dd49a7040643 Initial commit
fubar
parents:
diff changeset
6
dd49a7040643 Initial commit
fubar
parents:
diff changeset
7 <lifelines.StatisticalResult: logrank_test>
dd49a7040643 Initial commit
fubar
parents:
diff changeset
8 t_0 = -1
dd49a7040643 Initial commit
fubar
parents:
diff changeset
9 null_distribution = chi squared
dd49a7040643 Initial commit
fubar
parents:
diff changeset
10 degrees_of_freedom = 1
dd49a7040643 Initial commit
fubar
parents:
diff changeset
11 alpha = 0.99
dd49a7040643 Initial commit
fubar
parents:
diff changeset
12 test_name = logrank_test
dd49a7040643 Initial commit
fubar
parents:
diff changeset
13
dd49a7040643 Initial commit
fubar
parents:
diff changeset
14 ---
dd49a7040643 Initial commit
fubar
parents:
diff changeset
15 test_statistic p -log2(p)
dd49a7040643 Initial commit
fubar
parents:
diff changeset
16 0.58 0.45 1.16
dd49a7040643 Initial commit
fubar
parents:
diff changeset
17 ### Lifelines test of Proportional Hazards results with prio, age, race, paro, mar, fin as covariates on KM and CPH in lifelines test
dd49a7040643 Initial commit
fubar
parents:
diff changeset
18 <lifelines.CoxPHFitter: fitted with 432 total observations, 318 right-censored observations>
dd49a7040643 Initial commit
fubar
parents:
diff changeset
19 duration col = 'week'
dd49a7040643 Initial commit
fubar
parents:
diff changeset
20 event col = 'arrest'
dd49a7040643 Initial commit
fubar
parents:
diff changeset
21 baseline estimation = breslow
dd49a7040643 Initial commit
fubar
parents:
diff changeset
22 number of observations = 432
dd49a7040643 Initial commit
fubar
parents:
diff changeset
23 number of events observed = 114
dd49a7040643 Initial commit
fubar
parents:
diff changeset
24 partial log-likelihood = -659.00
dd49a7040643 Initial commit
fubar
parents:
diff changeset
25 time fit was run = 2023-08-09 07:43:37 UTC
dd49a7040643 Initial commit
fubar
parents:
diff changeset
26
dd49a7040643 Initial commit
fubar
parents:
diff changeset
27 ---
dd49a7040643 Initial commit
fubar
parents:
diff changeset
28 coef exp(coef) se(coef) coef lower 95% coef upper 95% exp(coef) lower 95% exp(coef) upper 95%
dd49a7040643 Initial commit
fubar
parents:
diff changeset
29 covariate
dd49a7040643 Initial commit
fubar
parents:
diff changeset
30 prio 0.10 1.10 0.03 0.04 0.15 1.04 1.16
dd49a7040643 Initial commit
fubar
parents:
diff changeset
31 age -0.06 0.94 0.02 -0.10 -0.02 0.90 0.98
dd49a7040643 Initial commit
fubar
parents:
diff changeset
32 race 0.32 1.38 0.31 -0.28 0.92 0.75 2.52
dd49a7040643 Initial commit
fubar
parents:
diff changeset
33 paro -0.09 0.91 0.20 -0.47 0.29 0.62 1.34
dd49a7040643 Initial commit
fubar
parents:
diff changeset
34 mar -0.48 0.62 0.38 -1.22 0.25 0.30 1.29
dd49a7040643 Initial commit
fubar
parents:
diff changeset
35 fin -0.38 0.68 0.19 -0.75 -0.00 0.47 1.00
dd49a7040643 Initial commit
fubar
parents:
diff changeset
36
dd49a7040643 Initial commit
fubar
parents:
diff changeset
37 cmp to z p -log2(p)
dd49a7040643 Initial commit
fubar
parents:
diff changeset
38 covariate
dd49a7040643 Initial commit
fubar
parents:
diff changeset
39 prio 0.00 3.53 <0.005 11.26
dd49a7040643 Initial commit
fubar
parents:
diff changeset
40 age 0.00 -2.95 <0.005 8.28
dd49a7040643 Initial commit
fubar
parents:
diff changeset
41 race 0.00 1.04 0.30 1.75
dd49a7040643 Initial commit
fubar
parents:
diff changeset
42 paro 0.00 -0.46 0.65 0.63
dd49a7040643 Initial commit
fubar
parents:
diff changeset
43 mar 0.00 -1.28 0.20 2.32
dd49a7040643 Initial commit
fubar
parents:
diff changeset
44 fin 0.00 -1.98 0.05 4.40
dd49a7040643 Initial commit
fubar
parents:
diff changeset
45 ---
dd49a7040643 Initial commit
fubar
parents:
diff changeset
46 Concordance = 0.63
dd49a7040643 Initial commit
fubar
parents:
diff changeset
47 Partial AIC = 1330.00
dd49a7040643 Initial commit
fubar
parents:
diff changeset
48 log-likelihood ratio test = 32.77 on 6 df
dd49a7040643 Initial commit
fubar
parents:
diff changeset
49 -log2(p) of ll-ratio test = 16.39
dd49a7040643 Initial commit
fubar
parents:
diff changeset
50
dd49a7040643 Initial commit
fubar
parents:
diff changeset
51
dd49a7040643 Initial commit
fubar
parents:
diff changeset
52 Bootstrapping lowess lines. May take a moment...
dd49a7040643 Initial commit
fubar
parents:
diff changeset
53
dd49a7040643 Initial commit
fubar
parents:
diff changeset
54
dd49a7040643 Initial commit
fubar
parents:
diff changeset
55 Bootstrapping lowess lines. May take a moment...
dd49a7040643 Initial commit
fubar
parents:
diff changeset
56
dd49a7040643 Initial commit
fubar
parents:
diff changeset
57 The ``p_value_threshold`` is set at 0.01. Even under the null hypothesis of no violations, some
dd49a7040643 Initial commit
fubar
parents:
diff changeset
58 covariates will be below the threshold by chance. This is compounded when there are many covariates.
dd49a7040643 Initial commit
fubar
parents:
diff changeset
59 Similarly, when there are lots of observations, even minor deviances from the proportional hazard
dd49a7040643 Initial commit
fubar
parents:
diff changeset
60 assumption will be flagged.
dd49a7040643 Initial commit
fubar
parents:
diff changeset
61
dd49a7040643 Initial commit
fubar
parents:
diff changeset
62 With that in mind, it's best to use a combination of statistical tests and visual tests to determine
dd49a7040643 Initial commit
fubar
parents:
diff changeset
63 the most serious violations. Produce visual plots using ``check_assumptions(..., show_plots=True)``
dd49a7040643 Initial commit
fubar
parents:
diff changeset
64 and looking for non-constant lines. See link [A] below for a full example.
dd49a7040643 Initial commit
fubar
parents:
diff changeset
65
dd49a7040643 Initial commit
fubar
parents:
diff changeset
66 <lifelines.StatisticalResult: proportional_hazard_test>
dd49a7040643 Initial commit
fubar
parents:
diff changeset
67 null_distribution = chi squared
dd49a7040643 Initial commit
fubar
parents:
diff changeset
68 degrees_of_freedom = 1
dd49a7040643 Initial commit
fubar
parents:
diff changeset
69 model = <lifelines.CoxPHFitter: fitted with 432 total observations, 318 right-censored observations>
dd49a7040643 Initial commit
fubar
parents:
diff changeset
70 test_name = proportional_hazard_test
dd49a7040643 Initial commit
fubar
parents:
diff changeset
71
dd49a7040643 Initial commit
fubar
parents:
diff changeset
72 ---
dd49a7040643 Initial commit
fubar
parents:
diff changeset
73 test_statistic p -log2(p)
dd49a7040643 Initial commit
fubar
parents:
diff changeset
74 age km 6.99 0.01 6.93
dd49a7040643 Initial commit
fubar
parents:
diff changeset
75 rank 7.40 0.01 7.26
dd49a7040643 Initial commit
fubar
parents:
diff changeset
76 fin km 0.02 0.90 0.15
dd49a7040643 Initial commit
fubar
parents:
diff changeset
77 rank 0.01 0.91 0.13
dd49a7040643 Initial commit
fubar
parents:
diff changeset
78 mar km 1.64 0.20 2.32
dd49a7040643 Initial commit
fubar
parents:
diff changeset
79 rank 1.80 0.18 2.48
dd49a7040643 Initial commit
fubar
parents:
diff changeset
80 paro km 0.06 0.81 0.31
dd49a7040643 Initial commit
fubar
parents:
diff changeset
81 rank 0.07 0.79 0.34
dd49a7040643 Initial commit
fubar
parents:
diff changeset
82 prio km 0.92 0.34 1.57
dd49a7040643 Initial commit
fubar
parents:
diff changeset
83 rank 0.88 0.35 1.52
dd49a7040643 Initial commit
fubar
parents:
diff changeset
84 race km 1.70 0.19 2.38
dd49a7040643 Initial commit
fubar
parents:
diff changeset
85 rank 1.68 0.19 2.36
dd49a7040643 Initial commit
fubar
parents:
diff changeset
86
dd49a7040643 Initial commit
fubar
parents:
diff changeset
87
dd49a7040643 Initial commit
fubar
parents:
diff changeset
88 1. Variable 'age' failed the non-proportional test: p-value is 0.0065.
dd49a7040643 Initial commit
fubar
parents:
diff changeset
89
dd49a7040643 Initial commit
fubar
parents:
diff changeset
90 Advice 1: the functional form of the variable 'age' might be incorrect. That is, there may be
dd49a7040643 Initial commit
fubar
parents:
diff changeset
91 non-linear terms missing. The proportional hazard test used is very sensitive to incorrect
dd49a7040643 Initial commit
fubar
parents:
diff changeset
92 functional forms. See documentation in link [D] below on how to specify a functional form.
dd49a7040643 Initial commit
fubar
parents:
diff changeset
93
dd49a7040643 Initial commit
fubar
parents:
diff changeset
94 Advice 2: try binning the variable 'age' using pd.cut, and then specify it in `strata=['age',
dd49a7040643 Initial commit
fubar
parents:
diff changeset
95 ...]` in the call in `.fit`. See documentation in link [B] below.
dd49a7040643 Initial commit
fubar
parents:
diff changeset
96
dd49a7040643 Initial commit
fubar
parents:
diff changeset
97 Advice 3: try adding an interaction term with your time variable. See documentation in link [C]
dd49a7040643 Initial commit
fubar
parents:
diff changeset
98 below.
dd49a7040643 Initial commit
fubar
parents:
diff changeset
99
dd49a7040643 Initial commit
fubar
parents:
diff changeset
100
dd49a7040643 Initial commit
fubar
parents:
diff changeset
101 Bootstrapping lowess lines. May take a moment...
dd49a7040643 Initial commit
fubar
parents:
diff changeset
102
dd49a7040643 Initial commit
fubar
parents:
diff changeset
103
dd49a7040643 Initial commit
fubar
parents:
diff changeset
104 Bootstrapping lowess lines. May take a moment...
dd49a7040643 Initial commit
fubar
parents:
diff changeset
105
dd49a7040643 Initial commit
fubar
parents:
diff changeset
106
dd49a7040643 Initial commit
fubar
parents:
diff changeset
107 Bootstrapping lowess lines. May take a moment...
dd49a7040643 Initial commit
fubar
parents:
diff changeset
108
dd49a7040643 Initial commit
fubar
parents:
diff changeset
109
dd49a7040643 Initial commit
fubar
parents:
diff changeset
110 Bootstrapping lowess lines. May take a moment...
dd49a7040643 Initial commit
fubar
parents:
diff changeset
111
dd49a7040643 Initial commit
fubar
parents:
diff changeset
112
dd49a7040643 Initial commit
fubar
parents:
diff changeset
113 ---
dd49a7040643 Initial commit
fubar
parents:
diff changeset
114 [A] https://lifelines.readthedocs.io/en/latest/jupyter_notebooks/Proportional%20hazard%20assumption.html
dd49a7040643 Initial commit
fubar
parents:
diff changeset
115 [B] https://lifelines.readthedocs.io/en/latest/jupyter_notebooks/Proportional%20hazard%20assumption.html#Bin-variable-and-stratify-on-it
dd49a7040643 Initial commit
fubar
parents:
diff changeset
116 [C] https://lifelines.readthedocs.io/en/latest/jupyter_notebooks/Proportional%20hazard%20assumption.html#Introduce-time-varying-covariates
dd49a7040643 Initial commit
fubar
parents:
diff changeset
117 [D] https://lifelines.readthedocs.io/en/latest/jupyter_notebooks/Proportional%20hazard%20assumption.html#Modify-the-functional-form
dd49a7040643 Initial commit
fubar
parents:
diff changeset
118 [E] https://lifelines.readthedocs.io/en/latest/jupyter_notebooks/Proportional%20hazard%20assumption.html#Stratification
dd49a7040643 Initial commit
fubar
parents:
diff changeset
119