@Qc@s dZddlZddlZddlZddlmZmZddlmZddl m Z m Z m Z m Z ddlmZdefdYZd efd YZd efd YZd efdYZdefdYZdefdYZdefdYZdS(s .. module:: wb_log_reader :synopsis: Reads and parses a log source to retrieve sets of records from it. This module defines several classes to handle MySQL server logs. It supports logs stored in the database as well as logs stored in files. All of the defined classes adhere to a common interface defining and implementing these public attributes and methods: Attributes: column_specs (tuple): Specifies each field in the log entries. The elements of this tuple are also tuples having the form (column_name, column_widh, [column_table_name]) where: column_name (str): A human readable name for the column. Frontend code should use this name wherever a column title is needed. column_width (int): The recommended with of the column column_table_name (str): (Optional) the name of the field referred by this column in the log table for DB logs partial_support: False if the log source is fully supported or a string explaining the limitations regarding the implemented log source reader class otherwise. Methods: has_previous(): Returns True if there are older entries that can be retrieved and False otherwise. has_next(): Returns True if there are newer entries that can be retrieved and False otherwise. first(): Returns a list of the first (oldest) records in the log. Each element in this list represents a single log entry and is also a list whose elements are the values for the columns defined in `column_specs`. last(): The same as `first()` but the records returned are the newest ones. previous(): Returns the records that precede the last retrieved records. Before calling it you should verify that `has_previous()` returns True. next(): Returns the records that follow the last retrieved records. Before calling it you should verify that `has_next()` returns True. current(): Returns the last retrieved records. range_text(): Returns a string that gives an indication of the position of the current records in the existent log set (if available). E.g. 'Records 1..50 of 145' refresh(): After calling this function the log reader should be able to manage new log entries that were added since the last call to this function or since the creation of the log reader object. This function doesn't return anything. If it is not possible to read the log entries, the class should raise an exception with a descriptive message to let the user know the reasons of the failure. Current limitations: ---------------------- * No remote server support for logs stored in files. * Cannot read files that aren't readable by the user running Workbench. iN(t WbAdminSSHtConnectionError(tPasswordHandler(tLogFileAccessErrort ServerIOErrortInvalidPasswordErrortUsers(tserver_os_pathtBaseQueryLogReadercBsqeZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z RS( so The base class for logs stored in a database. **This is not intended for direct instantiation.** cCs||_||_||_||_t|_d|_|jd|_t |j|jd|_ g|D]}|d^ql|_ dS(sConstructor :param ctrl_be: Control backend instance to make queries :param log_table: The name of the table where the log entries are stored :type log_table: str :param column_specs: Column definitions as explained in the module docstring :type column_specs: tuple :param ordering_column: The index for the column in `column_specs` that stores the timestamp of the log entries :type ordering_column: int ii2iN( t log_tabletctrl_bet column_specstordering_columntFalsetpartial_supportt total_counttrefresht show_counttmaxt show_starttcolnames(tselfR R R R tcolspec((s..\modules\wb_log_reader.pyt__init__ps        cCs |jdkS(Ni(R(R((s..\modules\wb_log_reader.pyt has_previousscCs|j|jdkS(Ni(RR(R((s..\modules\wb_log_reader.pythas_nextscCs |jS(N(t_query_records(R((s..\modules\wb_log_reader.pytcurrentscCs&t|j|jd|_|jS(Ni(RRRR(R((s..\modules\wb_log_reader.pytpreviousscCs)t|j|j|j|_|jS(N(tminRRRR(R((s..\modules\wb_log_reader.pytnextscCsd|_|jS(Ni(RR(R((s..\modules\wb_log_reader.pytfirsts cCs&t|j|jd|_|jS(Ni(RRRRR(R((s..\modules\wb_log_reader.pytlastscCs-d|jt|j|j|j|jfS(NsRecords %d..%d of %d(RRRR(R((s..\modules\wb_log_reader.pyt range_texts cCs|y|jjd|j}Wn#tk rB}td|nX| sW|j rftdn|jd|_dS(Ns SELECT count(*) AS count FROM %ssError fetching log contents: %ssError fetching log contentstcount(R t exec_queryR t ExceptionRtnextRowt intByNameR(Rtresultte((s..\modules\wb_log_reader.pyRscCsd|j|j|j|j|jf}y|jj|}Wn#tk rd}td|nXg}|rxd|j rg|jD]}|j |^q}|j |qtWn|j rt d|jn|S(Ns.SELECT * FROM %s ORDER BY %s DESC LIMIT %i, %isError fetching log contents: %ss1There were problems querying the server table %s.(R RR RRR R#R$RR%t stringByNametappendRtIOError(RtqueryR'R(trecordstcolnametrow((s..\modules\wb_log_reader.pyRs"  % (t__name__t __module__t__doc__RRRRRRRR R!RR(((s..\modules\wb_log_reader.pyRjs          tGeneralQueryLogReadercBseZddZRS(smysql.general_logcCsDddddddf}d|_tt|j|||ddS(NtTimeit event_timetFromixt user_hosttThreadiPt thread_idtServert server_ids Command Typet command_typetDetailitargumentii(R4iR5(R6ixR7(R8iPR9(R:iPR;(s Command TypeiPR<(R=iR>(t detail_columntsuperR3R(RR t table_nameR ((s..\modules\wb_log_reader.pyRs  (R0R1R(((s..\modules\wb_log_reader.pyR3stSlowQueryLogReadercBseZddZRS(smysql.slow_logc CsSddd d!d"d#d$d%d&d'd(f }d|_tt|j|||ddS()Ns Start Timeit start_timeR6ixR7s Query Timet query_times Lock Timet lock_times Rows Senti2t rows_sents Rows Examinedt rows_examinedtDBiPtdbsLast Insert IDtlast_insert_ids Insert IDt insert_ids Server IDR;tSQLitsql_texti i(s Start TimeiRC(sFromixs user_host(s Query TimeiRD(s Lock TimeiRE(s Rows Senti2RF(s Rows Examinedi2RG(RHiPRI(sLast Insert IDi2RJ(s Insert IDi2RK(s Server IDi2s server_id(RLiRM(R?R@RBR(RR RAR ((s..\modules\wb_log_reader.pyRs  (R0R1R(((s..\modules\wb_log_reader.pyRBstBaseLogFileReadercBseZdZedZdZdZdZdZdZ dZ dZ d Z d Z d Zd Zd ZdZdZdZdZdZdZRS(sj The base class for logs stored in files. **This is not intended for direct instantiation.** c Cs||_||_||_||_|jjj|_t|_t|_d|_ t |t t j fr||_t|dr|j|_tj|jj|_ q%d|_nq|jd}t|jj}|jjj}| r |jr |jjd}n|j|r!|n|j|||_|jry1tj|jj|_ t|jd|_Wq%tk r} |jjjrtd|jntj j!|jrwt"|_tj j#|j|_$|j%j&d|_'y9|j(d||jfdt)j*d |j'd |_+Wn$t,k rQ|j%j-dnXt j |j+|_d ||_qt| q%t.k r} td |jtj j/|jfq%Xna|jjj0std n|jjj1stdn|jj2r|jj2|_2ntt3|_2xet"ry)|j2j4|jjt5|jjWn.t6k r} t7| j8dsqq-XPq-W|j2j9stdn|j2j:j;|_<y7|j<j|j|_|j<j|jj|_ Wn&tk r$} td|jnX|jj=d d|jj>|_?||_@|j?|_Ad |_B|jC|jB|_D|j?|_BdS(sgConstructor :param ctrl_be: Control backend instance to retrieve root password if needed :param log_file: The path to the log file to read from or a file like instance to read log entries from :type log_file: str/file :param pat: A regular expression pattern that matches a log entry :type pat: regular expression object :param chunk_size: The size in bytes of the chunks that are read from the log file :type chunk_size: int :param truncate_long_lines: Whether the log entries that are long should be abbreviated :type truncate_long_lines: bool :param append_gaps: Whether the data between the end of the record regex and the start of the next record regex should be added to the previous entry :type append_gaps: bool tnames "tdatadirtrbs1Cannot read from %s. Check permissions and retry.tfiles tail -c %d %stas_usert user_passwordisThe current user has no permission to read %s. Reading it with superuser privileges. Workbench has only limited support for this.sAn error appeared when looking for the log file "%s". Please verify that you can access the directory where the log file is placed ("%s") and that the log file is actually there. Try again when done.sYou have not enabled remote administration for this server. Without it this log file cannot be shown. Please enable remote administration in this server instance and try again.siRemote log files are only supported for SSH connection. Please configure an SSH connection and try again.s9Could not establish SSH connection: Authentication faileds"Could not connect to remote serversKCould not read file %s in remote server. Please verify path and permissionsiN(Etpatt append_gapsttruncate_long_linesR tserver_profiletis_localR t not_readableRtNonetmtimet isinstanceRRtStringIOtlog_filethasattrROt log_file_nametoststattst_mtimetstripRRPtis_sql_connectedtget_server_variabletisabstjointopenR+ttarget_is_windowsRtpathtisfiletTruetgetsizetlast_known_sizetpassword_handlertget_password_fortpasswordtexecute_filtered_commandRtADMINt tail_dataRtreset_password_fortOSErrortdirnametremote_admin_enabledtuses_sshtsshRtwrapped_connectRRtstrt startswitht is_connectedtclientt open_sftptsftptseekttellt file_sizet chunk_sizet chunk_startt chunk_endt_adjust_chunk_endtfirst_record_pos( RR R_RURRWRVtospathRPR(terror((s..\modules\wb_log_reader.pyRs          *      ,           cCs|j|jkS(s@ If there is a previous chunk that can be read. (RR(R((s..\modules\wb_log_reader.pyRcscCs|j|jkS(s< If there is a next chunk that can be read. (RR(R((s..\modules\wb_log_reader.pyRiscCsG|jsd|j|jSd|j|j|j|jfSdS(NsLog file size: %ss$Log file size: %s. (Reading last %s)(RZt _format_sizeRRp(R((s..\modules\wb_log_reader.pyR!os cCs|dkrdSd dddf}xTt|D]F\}}||dkr/d |||d d||d d fSq/Wd ||d d|d d fS(sV Returns a string with a human friendly representation of a file size is0 Bg?tBg@tkBitMBtGBs%.1f %sii(g?R(g@Rg0A(g0ARigA(gAR(t enumerate(Rtbytestunitstidxtunit((s..\modules\wb_log_reader.pyRvs  .cCs}|j|jkr!tdn|j|_|jt|j|j|j|_|j|jj |j|j S(s+ Reads the previous chunk. s"No more data to read from log file( RRt IndexErrorRRRRt_adjust_chunk_startR_Rt _read_chunk(R((s..\modules\wb_log_reader.pyt_read_previous_chunks   cCsn|j|jkr!tdn|j|_t|j|jd|j|_|j|j|jS(s' Reads the next chunk. s"No more data to read from log filei( RRRRRRRRR(R((s..\modules\wb_log_reader.pyt_read_next_chunks #  cCs9|j|j}|jj|j|jj|}|S(s* Reads the current chunk. (RRR_Rtread(RRtdata((s..\modules\wb_log_reader.pyRscCs|j|jkr|j|jkr4|j|_dSd}|j|j}xtr|jj|j|jj||}|jj|}|rPn|j|jkrt dnt |j d|j|j}|j|8_qMWndS(s Puts the chunk start marker at the beginning of a record. If a record starts within the range [self.chunk_start, self.chunk_end] the chunk start marker will point to it. If there's no record in the range, move backwards until there's at least one record in the chunk. Precondition: the chunk end marker must be set to delimit the end of the chunk. Nts"No more data to read from log filei( RRRRnR_RRRUtsearchRRR(RRRtpat_occurrence((s..\modules\wb_log_reader.pyRs    cCs|j|jkrd}|jj|j|jd}xTtr||jj|7}|jj|}|s|jj |jkr;Pq;q;W|r|j|j n|j|_ndS(s Moves the chunk termination marker forward to just before the start of the next record or to the EOF if not next record. RiN( RRR_RRRnRRURRtstart(Rt trailing_dataRR((s..\modules\wb_log_reader.pyRs  cCshg}|jj|}|rQ|j}|j|jkrQ|j|7_qQnx|r |j}||dkr|jr|ddc|||!7    ( 7         (R0R1R2RnRRRR!RRRRRRRRRRRRR RR(((s..\modules\wb_log_reader.pyRNs( u          tErrorLogFileReadercBseZdZdedZRS(sX This class enables the retrieval of log entries in a MySQL error log file. iicCsStjdtj}tt|j|||||ddf|_d|_dS( Ns(^(\d{6} {1,2}\d{1,2}:\d{2}:\d{2}) (.*?)$R4itDetailsii(sTimei(Ri(tretcompiletMR@RRR R?(RR t file_nameRRWRU((s..\modules\wb_log_reader.pyRls " i(R0R1R2R R(((s..\modules\wb_log_reader.pyRgstGeneralLogFileReadercBseZdZdedZRS(s` This class enables the retrieval of log entries in a MySQL general query log file. iicCsYtjdtj}tt|j|||||d d d d f|_d |_dS(NsQ^(\d{6} {1,2}\d{1,2}:\d{2}:\d{2}[\t ]+|[\t ]+)(\s*\d+)(\s*.*?)(?:\t+| {2,})(.*?)$R4iR8iPs Command TypeR=ii(sTimei(sThreadiP(s Command TypeiP(sDetaili(RRRR@RRR R?(RR RRRWRU((s..\modules\wb_log_reader.pyR{s" i(R0R1R2RnR(((s..\modules\wb_log_reader.pyRvstSlowLogFileReadercBs eZdZdeedZRS(s] This class enables the retrieval of log entries in a MySQL slow query log file. iicCsetjdtj}tt|j||||||d ddddddf|_d |_dS(Ns(?:^|\n)# Time: (\d{6} {1,2}\d{1,2}:\d{2}:\d{2}).*?\n# User@Host: (.*?)\n# Query_time: +([0-9.]+) +Lock_time: +([\d.]+) +Rows_sent: +(\d+) +Rows_examined: +(\d+)\n(.*?)(?=\n# |\n[^\n]+, Version: |$)s Start Timeis User@HostiPs Query Times Lock Times Rows Sents Rows ExaminedR=ii(s Start Timei(s User@HostiP(s Query TimeiP(s Lock TimeiP(s Rows SentiP(s Rows ExaminediP(sDetaili(RRtSR@RRR R?(RR RRRWRVRU((s..\modules\wb_log_reader.pyRs% i(R0R1R2R R(((s..\modules\wb_log_reader.pyRs(R2RRbR^t wb_admin_sshRRtwb_server_controlRt wb_commonRRRRtworkbench.utilsRtobjectRR3RBRNRRR(((s..\modules\wb_log_reader.pyt\s   "\